Computer vision & CNNs: Cats vs. Dogs

In this example, we are going to apply a CNN to classify dogs vs. cats images. This will walk you through the fundamentals of importing images, applying image augmentation, and performing classification on them.

Learning objectives:

• What image generators are, why and how to use them.
• What image augmentation is, why and how to use them.

Required packages

library(keras)
library(glue)
library(tidyverse)

Registered S3 methods overwritten by 'dbplyr':
  method         from
  print.tbl_lazy     
  print.tbl_sql      
── Attaching packages ─────────────────────────────────────────────────────── tidyverse 1.3.1 ──
✓ ggplot2 3.3.5     ✓ purrr   0.3.4
✓ tibble  3.1.5     ✓ dplyr   1.0.7
✓ tidyr   1.1.4     ✓ stringr 1.4.0
✓ readr   2.0.2     ✓ forcats 0.5.1
── Conflicts ────────────────────────────────────────────────────────── tidyverse_conflicts() ──
x dplyr::collapse() masks glue::collapse()
x dplyr::filter()   masks stats::filter()
x dplyr::lag()      masks stats::lag()

Data Preparation

Image location

We are going to use the Dogs vs. Cats Kaggle competition data set (https://www.kaggle.com/c/dogs-vs-cats/data). However, do to size and runtime limitations, we are going to only use a subset of the data. We have already set up the directories which look like:

paste0(data_directory,"dogs-vs-cats.zip")

[1] "/Users/milou/Documents/misk_projects/misk-dl/materials/datadogs-vs-cats.zip"

- data
   └── dogs-vs-cats
       └── train
           └── cats
               ├── cat.1.jpg
               ├── cat.2.jpg
               └── ...
           └── dogs
               ├── dog.1.jpg
               ├── dog.2.jpg
               └── ...
       └── validation
           ├── cats
           └── dogs
       └── test
           ├── cats
           └── dogs

# define the directories:
image_dir <- here::here("materials", "data", "dogs-vs-cats")

train_dir <- file.path(image_dir, "train")
valid_dir <- file.path(image_dir, "validation")
test_dir <- file.path(image_dir, "test")

# create train, validation, and test file paths for cat images
train_cats_dir <- file.path(train_dir, "cats")
valid_cats_dir <- file.path(valid_dir, "cats")
test_cats_dir <- file.path(test_dir, "cats")

# create train, validation, and test file paths for dog images
train_dogs_dir <- file.path(train_dir, "dogs")
valid_dogs_dir <- file.path(valid_dir, "dogs")
test_dogs_dir <- file.path(test_dir, "dogs")

Data set

Although there are 25,000 images in this data set, we are going to use a very small subset, which includes:

glue("Cat images:",
     " - total training cat images: {length(list.files(train_cats_dir))}",
     " - total validation cat images: {length(list.files(valid_cats_dir))}",
     " - total test cat images: {length(list.files(test_cats_dir))}",
     "\n",
     "Dog images:",
     " - total training dog images: {length(list.files(train_dogs_dir))}",
     " - total validation dog images: {length(list.files(valid_dogs_dir))}",
     " - total test dog images: {length(list.files(test_dogs_dir))}",
     .sep = "\n"
     )

Cat images:
 - total training cat images: 0
 - total validation cat images: 0
 - total test cat images: 0


Dog images:
 - total training dog images: 0
 - total validation dog images: 0
 - total test dog images: 0

glue("Cat images:",
     " - total training cat images: {length(list.files(train_cats_dir))}",
     " - total validation cat images: {length(list.files(valid_cats_dir))}",
     " - total test cat images: {length(list.files(test_cats_dir))}",
     "\n",
     "Dog images:",
     " - total training dog images: {length(list.files(train_dogs_dir))}",
     " - total validation dog images: {length(list.files(valid_dogs_dir))}",
     " - total test dog images: {length(list.files(test_dogs_dir))}",
     .sep = "\n"
     )

Cat images:
 - total training cat images: 1000
 - total validation cat images: 500
 - total test cat images: 500


Dog images:
 - total training dog images: 1000
 - total validation dog images: 500
 - total test dog images: 500

Let’s check out the first 10 cat and dog images:

op <- par(mfrow = c(4, 5), pty = "s", mar = c(0.1, 0.1, 0.1, 0.1))
for (i in 1:10) {
  plot(as.raster(jpeg::readJPEG(paste0(train_cats_dir, "/cat.", i, ".jpg"))))
  plot(as.raster(jpeg::readJPEG(paste0(train_dogs_dir, "/dog.", i, ".jpg"))))
}
par(op)

CNN with image generator

Define and compile model

We’re going to set up a simple CNN model that contains steps you saw in the previous module. This CNN includes:

• Four sequential conv and max pooling layers
• Flatten layer
• Densly-connected network
• Single binary output

model <- keras_model_sequential() %>%
  
  # feature detector portion of model
  layer_conv_2d(filters = 32, kernel_size = c(3, 3), activation = "relu", 
                input_shape = c(150, 150, 3)) %>%
  layer_max_pooling_2d(pool_size = c(2, 2)) %>%
  
  layer_conv_2d(filters = 64, kernel_size = c(3, 3), activation = "relu") %>% 
  layer_max_pooling_2d(pool_size = c(2, 2)) %>%
  
  layer_conv_2d(filters = 128, kernel_size = c(3, 3), activation = "relu") %>% 
  layer_max_pooling_2d(pool_size = c(2, 2)) %>%
  
  layer_conv_2d(filters = 128, kernel_size = c(3, 3), activation = "relu") %>% 
  layer_max_pooling_2d(pool_size = c(2, 2)) %>%
  
  # classifier portion of model
  layer_flatten() %>%
  layer_dense(units = 512, activation = "relu") %>%
  layer_dense(units = 1, activation = "sigmoid")

summary(model)

Model: "sequential_1"
_____________________________________________________________________________________________________________________________
Layer (type)                                            Output Shape                                      Param #            
=============================================================================================================================
conv2d_6 (Conv2D)                                       (None, 148, 148, 32)                              896                
_____________________________________________________________________________________________________________________________
max_pooling2d_5 (MaxPooling2D)                          (None, 74, 74, 32)                                0                  
_____________________________________________________________________________________________________________________________
conv2d_5 (Conv2D)                                       (None, 72, 72, 64)                                18496              
_____________________________________________________________________________________________________________________________
max_pooling2d_4 (MaxPooling2D)                          (None, 36, 36, 64)                                0                  
_____________________________________________________________________________________________________________________________
conv2d_4 (Conv2D)                                       (None, 34, 34, 128)                               73856              
_____________________________________________________________________________________________________________________________
max_pooling2d_3 (MaxPooling2D)                          (None, 17, 17, 128)                               0                  
_____________________________________________________________________________________________________________________________
conv2d_3 (Conv2D)                                       (None, 15, 15, 128)                               147584             
_____________________________________________________________________________________________________________________________
max_pooling2d_2 (MaxPooling2D)                          (None, 7, 7, 128)                                 0                  
_____________________________________________________________________________________________________________________________
flatten_1 (Flatten)                                     (None, 6272)                                      0                  
_____________________________________________________________________________________________________________________________
dense_3 (Dense)                                         (None, 512)                                       3211776            
_____________________________________________________________________________________________________________________________
dense_2 (Dense)                                         (None, 1)                                         513                
=============================================================================================================================
Total params: 3,453,121
Trainable params: 3,453,121
Non-trainable params: 0
_____________________________________________________________________________________________________________________________

Compile the model:

model %>% compile(
  loss = "binary_crossentropy",
  optimizer = optimizer_rmsprop(lr = 0.0001),
  metrics = "accuracy"
)

Read images from directories

Next, we need a process that imports our images and transforms them to tensors that our model can process. We’ll use two functions to perform this process.

image_data_generator will:

1. Read the image files
2. Decode the image to RGB grids of pixels
3. Convert these into floating point tensors
4. Rescale pixel values to [0, 1] interval

image_data_generator provides other capabilities that we’ll look at shortly.

flow_images_from_directory will:

1. Apply image_data_generator
2. To a batch of 20 images at a time
3. From our training directory (randomly shuffling between subdirectories)
4. Resize these images to be consistent size of 150x150 pixels
5. Apply binary labels

train_datagen <- image_data_generator(rescale = 1/255)
valid_datagen <- image_data_generator(rescale = 1/255)

train_generator <- flow_images_from_directory(
  train_dir,
  train_datagen,
  target_size = c(150, 150),
  batch_size = 20,
  class_mode = "binary"
)

Found 2000 images belonging to 2 classes.

validation_generator <- flow_images_from_directory(
  valid_dir,
  valid_datagen,
  target_size = c(150, 150),
  batch_size = 20,
  class_mode = "binary"
)

Found 1000 images belonging to 2 classes.

If we get the first batch from the generator, you will see that it yields 20 images of 150x150 pixels with three channels (20, 150, 150, 3) along with their binary labels (0, 1).

batch <- generator_next(train_generator)
str(batch)

List of 2
 $ : num [1:20, 1:150, 1:150, 1:3] 0.8118 0.6667 0.0235 0.1647 0.2824 ...
 $ : num [1:20(1d)] 0 1 1 0 1 1 0 1 1 0 ...

Train the model

To train our model we’ll use fit_generator which is the equivalent of fit for data generators. We provide it our generators for the training and validation data. Plus, we need to specify:

• steps_per_epoch: how many samples to draw from the training generator before declaring an epoch over. Our generator supplies batches of 20 and we have 2,000 training images so we need 100 steps.
• validation_steps: how many samples to draw from the validation generator. Our generator supplies batches of 20 and we have 1,000 validation images so we need 50 steps.

Note:

• Without a GPU this will take approximately 20 minutes to train
• With GPUs this will take approximately 5 minutes

history <- model %>% fit_generator(
  train_generator,
  steps_per_epoch = 100,
  epochs = 30,
  validation_data = validation_generator,
  validation_steps = 50,
  callbacks = callback_early_stopping(patience = 5)
)

Epoch 1/30

  1/100 [..............................] - ETA: 2:39 - loss: 0.6953 - accuracy: 0.5000
  2/100 [..............................] - ETA: 41s - loss: 0.6975 - accuracy: 0.5000 
  3/100 [..............................] - ETA: 38s - loss: 0.6838 - accuracy: 0.5333
  4/100 [>.............................] - ETA: 40s - loss: 0.6855 - accuracy: 0.5500
  5/100 [>.............................] - ETA: 40s - loss: 0.6862 - accuracy: 0.5400
  6/100 [>.............................] - ETA: 39s - loss: 0.6855 - accuracy: 0.5500
  7/100 [=>............................] - ETA: 37s - loss: 0.6831 - accuracy: 0.5786
  8/100 [=>............................] - ETA: 37s - loss: 0.6984 - accuracy: 0.5562
  9/100 [=>............................] - ETA: 35s - loss: 0.6978 - accuracy: 0.5500
 10/100 [==>...........................] - ETA: 35s - loss: 0.6968 - accuracy: 0.5600
 11/100 [==>...........................] - ETA: 34s - loss: 0.6960 - accuracy: 0.5591
 12/100 [==>...........................] - ETA: 33s - loss: 0.6945 - accuracy: 0.5583
 13/100 [==>...........................] - ETA: 33s - loss: 0.6955 - accuracy: 0.5462
 14/100 [===>..........................] - ETA: 32s - loss: 0.6952 - accuracy: 0.5429
 15/100 [===>..........................] - ETA: 32s - loss: 0.6955 - accuracy: 0.5333
 16/100 [===>..........................] - ETA: 31s - loss: 0.6954 - accuracy: 0.5281
 17/100 [====>.........................] - ETA: 31s - loss: 0.6953 - accuracy: 0.5265
 18/100 [====>.........................] - ETA: 31s - loss: 0.6951 - accuracy: 0.5250
 19/100 [====>.........................] - ETA: 30s - loss: 0.6961 - accuracy: 0.5132
 20/100 [=====>........................] - ETA: 30s - loss: 0.6977 - accuracy: 0.5050
 21/100 [=====>........................] - ETA: 29s - loss: 0.6972 - accuracy: 0.5119
 22/100 [=====>........................] - ETA: 29s - loss: 0.6969 - accuracy: 0.5182
 23/100 [=====>........................] - ETA: 29s - loss: 0.6964 - accuracy: 0.5217
 24/100 [======>.......................] - ETA: 28s - loss: 0.6960 - accuracy: 0.5250
 25/100 [======>.......................] - ETA: 28s - loss: 0.6962 - accuracy: 0.5180
 26/100 [======>.......................] - ETA: 27s - loss: 0.6958 - accuracy: 0.5192
 27/100 [=======>......................] - ETA: 27s - loss: 0.6951 - accuracy: 0.5204
 28/100 [=======>......................] - ETA: 27s - loss: 0.6956 - accuracy: 0.5179
 29/100 [=======>......................] - ETA: 26s - loss: 0.6958 - accuracy: 0.5138
 30/100 [========>.....................] - ETA: 26s - loss: 0.6961 - accuracy: 0.5050
 31/100 [========>.....................] - ETA: 26s - loss: 0.6956 - accuracy: 0.5081
 32/100 [========>.....................] - ETA: 25s - loss: 0.6957 - accuracy: 0.5078
 33/100 [========>.....................] - ETA: 25s - loss: 0.6954 - accuracy: 0.5121
 34/100 [=========>....................] - ETA: 25s - loss: 0.6950 - accuracy: 0.5191
 35/100 [=========>....................] - ETA: 24s - loss: 0.6942 - accuracy: 0.5214
 36/100 [=========>....................] - ETA: 24s - loss: 0.6946 - accuracy: 0.5181
 37/100 [==========>...................] - ETA: 24s - loss: 0.6944 - accuracy: 0.5176
 38/100 [==========>...................] - ETA: 23s - loss: 0.6945 - accuracy: 0.5145
 39/100 [==========>...................] - ETA: 23s - loss: 0.6941 - accuracy: 0.5205
 40/100 [===========>..................] - ETA: 23s - loss: 0.6943 - accuracy: 0.5200
 41/100 [===========>..................] - ETA: 22s - loss: 0.6941 - accuracy: 0.5195
 42/100 [===========>..................] - ETA: 22s - loss: 0.6935 - accuracy: 0.5238
 43/100 [===========>..................] - ETA: 21s - loss: 0.6941 - accuracy: 0.5221
 44/100 [============>.................] - ETA: 21s - loss: 0.6935 - accuracy: 0.5250
 45/100 [============>.................] - ETA: 21s - loss: 0.6940 - accuracy: 0.5233
 46/100 [============>.................] - ETA: 20s - loss: 0.6935 - accuracy: 0.5250
 47/100 [=============>................] - ETA: 20s - loss: 0.6938 - accuracy: 0.5234
 48/100 [=============>................] - ETA: 19s - loss: 0.6940 - accuracy: 0.5208
 49/100 [=============>................] - ETA: 19s - loss: 0.6939 - accuracy: 0.5204
 50/100 [==============>...............] - ETA: 18s - loss: 0.6937 - accuracy: 0.5240
 51/100 [==============>...............] - ETA: 18s - loss: 0.6936 - accuracy: 0.5225
 52/100 [==============>...............] - ETA: 18s - loss: 0.6937 - accuracy: 0.5192
 53/100 [==============>...............] - ETA: 17s - loss: 0.6937 - accuracy: 0.5189
 54/100 [===============>..............] - ETA: 17s - loss: 0.6935 - accuracy: 0.5222
 55/100 [===============>..............] - ETA: 17s - loss: 0.6936 - accuracy: 0.5209
 56/100 [===============>..............] - ETA: 16s - loss: 0.6939 - accuracy: 0.5170
 57/100 [================>.............] - ETA: 16s - loss: 0.6939 - accuracy: 0.5158
 58/100 [================>.............] - ETA: 16s - loss: 0.6938 - accuracy: 0.5155
 59/100 [================>.............] - ETA: 15s - loss: 0.6936 - accuracy: 0.5161
 60/100 [=================>............] - ETA: 15s - loss: 0.6927 - accuracy: 0.5200
 61/100 [=================>............] - ETA: 14s - loss: 0.6923 - accuracy: 0.5197
 62/100 [=================>............] - ETA: 14s - loss: 0.6932 - accuracy: 0.5177
 63/100 [=================>............] - ETA: 14s - loss: 0.6932 - accuracy: 0.5175
 64/100 [==================>...........] - ETA: 13s - loss: 0.6932 - accuracy: 0.5156
 65/100 [==================>...........] - ETA: 13s - loss: 0.6931 - accuracy: 0.5177
 66/100 [==================>...........] - ETA: 12s - loss: 0.6930 - accuracy: 0.5174
 67/100 [===================>..........] - ETA: 12s - loss: 0.6927 - accuracy: 0.5201
 68/100 [===================>..........] - ETA: 12s - loss: 0.6931 - accuracy: 0.5176
 69/100 [===================>..........] - ETA: 11s - loss: 0.6932 - accuracy: 0.5174
 70/100 [====================>.........] - ETA: 11s - loss: 0.6929 - accuracy: 0.5186
 71/100 [====================>.........] - ETA: 10s - loss: 0.6921 - accuracy: 0.5211
 72/100 [====================>.........] - ETA: 10s - loss: 0.6936 - accuracy: 0.5174
 73/100 [====================>.........] - ETA: 10s - loss: 0.6935 - accuracy: 0.5171
 74/100 [=====================>........] - ETA: 9s - loss: 0.6934 - accuracy: 0.5162 
 75/100 [=====================>........] - ETA: 9s - loss: 0.6933 - accuracy: 0.5173
 76/100 [=====================>........] - ETA: 9s - loss: 0.6930 - accuracy: 0.5197
 77/100 [======================>.......] - ETA: 8s - loss: 0.6928 - accuracy: 0.5208
 78/100 [======================>.......] - ETA: 8s - loss: 0.6924 - accuracy: 0.5237
 79/100 [======================>.......] - ETA: 7s - loss: 0.6927 - accuracy: 0.5222
 80/100 [=======================>......] - ETA: 7s - loss: 0.6923 - accuracy: 0.5244
 81/100 [=======================>......] - ETA: 7s - loss: 0.6919 - accuracy: 0.5235
 82/100 [=======================>......] - ETA: 6s - loss: 0.6917 - accuracy: 0.5250
 83/100 [=======================>......] - ETA: 6s - loss: 0.6914 - accuracy: 0.5265
 84/100 [========================>.....] - ETA: 5s - loss: 0.6911 - accuracy: 0.5286
 85/100 [========================>.....] - ETA: 5s - loss: 0.6907 - accuracy: 0.5306
 86/100 [========================>.....] - ETA: 5s - loss: 0.6907 - accuracy: 0.5302
 87/100 [=========================>....] - ETA: 4s - loss: 0.6906 - accuracy: 0.5316
 88/100 [=========================>....] - ETA: 4s - loss: 0.6901 - accuracy: 0.5335
 89/100 [=========================>....] - ETA: 4s - loss: 0.6901 - accuracy: 0.5331
 90/100 [==========================>...] - ETA: 3s - loss: 0.6899 - accuracy: 0.5339
 91/100 [==========================>...] - ETA: 3s - loss: 0.6894 - accuracy: 0.5352
 92/100 [==========================>...] - ETA: 2s - loss: 0.6899 - accuracy: 0.5342
 93/100 [==========================>...] - ETA: 2s - loss: 0.6898 - accuracy: 0.5339
 94/100 [===========================>..] - ETA: 2s - loss: 0.6897 - accuracy: 0.5324
 95/100 [===========================>..] - ETA: 1s - loss: 0.6895 - accuracy: 0.5332
 96/100 [===========================>..] - ETA: 1s - loss: 0.6892 - accuracy: 0.5339
 97/100 [============================>.] - ETA: 1s - loss: 0.6890 - accuracy: 0.5345
 98/100 [============================>.] - ETA: 0s - loss: 0.6885 - accuracy: 0.5362
 99/100 [============================>.] - ETA: 0s - loss: 0.6876 - accuracy: 0.5384
100/100 [==============================] - 39s 375ms/step - loss: 0.6877 - accuracy: 0.5390

100/100 [==============================] - 44s 425ms/step - loss: 0.6877 - accuracy: 0.5390 - val_loss: 0.6844 - val_accuracy: 0.5080
Epoch 2/30

  1/100 [..............................] - ETA: 1:03 - loss: 0.7128 - accuracy: 0.5000
  2/100 [..............................] - ETA: 40s - loss: 0.6903 - accuracy: 0.4750 
  3/100 [..............................] - ETA: 43s - loss: 0.6843 - accuracy: 0.5500
  4/100 [>.............................] - ETA: 41s - loss: 0.7032 - accuracy: 0.5000
  5/100 [>.............................] - ETA: 38s - loss: 0.6987 - accuracy: 0.5100
  6/100 [>.............................] - ETA: 37s - loss: 0.6971 - accuracy: 0.4917
  7/100 [=>............................] - ETA: 36s - loss: 0.6912 - accuracy: 0.5286
  8/100 [=>............................] - ETA: 35s - loss: 0.6923 - accuracy: 0.5250
  9/100 [=>............................] - ETA: 35s - loss: 0.6905 - accuracy: 0.5333
 10/100 [==>...........................] - ETA: 34s - loss: 0.6922 - accuracy: 0.5300
 11/100 [==>...........................] - ETA: 34s - loss: 0.6899 - accuracy: 0.5364
 12/100 [==>...........................] - ETA: 33s - loss: 0.6899 - accuracy: 0.5250
 13/100 [==>...........................] - ETA: 33s - loss: 0.6893 - accuracy: 0.5231
 14/100 [===>..........................] - ETA: 32s - loss: 0.6872 - accuracy: 0.5286
 15/100 [===>..........................] - ETA: 32s - loss: 0.6850 - accuracy: 0.5433
 16/100 [===>..........................] - ETA: 32s - loss: 0.6842 - accuracy: 0.5469
 17/100 [====>.........................] - ETA: 32s - loss: 0.6817 - accuracy: 0.5559
 18/100 [====>.........................] - ETA: 31s - loss: 0.6777 - accuracy: 0.5611
 19/100 [====>.........................] - ETA: 31s - loss: 0.6800 - accuracy: 0.5526
 20/100 [=====>........................] - ETA: 31s - loss: 0.6824 - accuracy: 0.5475
 21/100 [=====>........................] - ETA: 30s - loss: 0.6809 - accuracy: 0.5548
 22/100 [=====>........................] - ETA: 29s - loss: 0.6789 - accuracy: 0.5568
 23/100 [=====>........................] - ETA: 29s - loss: 0.6779 - accuracy: 0.5630
 24/100 [======>.......................] - ETA: 29s - loss: 0.6769 - accuracy: 0.5688
 25/100 [======>.......................] - ETA: 28s - loss: 0.6751 - accuracy: 0.5740
 26/100 [======>.......................] - ETA: 28s - loss: 0.6738 - accuracy: 0.5808
 27/100 [=======>......................] - ETA: 27s - loss: 0.6726 - accuracy: 0.5833
 28/100 [=======>......................] - ETA: 27s - loss: 0.6727 - accuracy: 0.5821
 29/100 [=======>......................] - ETA: 26s - loss: 0.6766 - accuracy: 0.5759
 30/100 [========>.....................] - ETA: 26s - loss: 0.6751 - accuracy: 0.5800
 31/100 [========>.....................] - ETA: 25s - loss: 0.6736 - accuracy: 0.5823
 32/100 [========>.....................] - ETA: 25s - loss: 0.6714 - accuracy: 0.5859
 33/100 [========>.....................] - ETA: 25s - loss: 0.6698 - accuracy: 0.5894
 34/100 [=========>....................] - ETA: 24s - loss: 0.6724 - accuracy: 0.5853
 35/100 [=========>....................] - ETA: 24s - loss: 0.6707 - accuracy: 0.5900
 36/100 [=========>....................] - ETA: 23s - loss: 0.6689 - accuracy: 0.5917
 37/100 [==========>...................] - ETA: 23s - loss: 0.6640 - accuracy: 0.5973
 38/100 [==========>...................] - ETA: 23s - loss: 0.6647 - accuracy: 0.5947
 39/100 [==========>...................] - ETA: 22s - loss: 0.6645 - accuracy: 0.5949
 40/100 [===========>..................] - ETA: 22s - loss: 0.6658 - accuracy: 0.5925
 41/100 [===========>..................] - ETA: 22s - loss: 0.6645 - accuracy: 0.5939
 42/100 [===========>..................] - ETA: 21s - loss: 0.6663 - accuracy: 0.5917
 43/100 [===========>..................] - ETA: 21s - loss: 0.6652 - accuracy: 0.5930
 44/100 [============>.................] - ETA: 21s - loss: 0.6644 - accuracy: 0.5955
 45/100 [============>.................] - ETA: 20s - loss: 0.6645 - accuracy: 0.5956
 46/100 [============>.................] - ETA: 20s - loss: 0.6650 - accuracy: 0.5967
 47/100 [=============>................] - ETA: 19s - loss: 0.6654 - accuracy: 0.5968
 48/100 [=============>................] - ETA: 19s - loss: 0.6650 - accuracy: 0.5969
 49/100 [=============>................] - ETA: 19s - loss: 0.6644 - accuracy: 0.5980
 50/100 [==============>...............] - ETA: 18s - loss: 0.6648 - accuracy: 0.5970
 51/100 [==============>...............] - ETA: 18s - loss: 0.6654 - accuracy: 0.5941
 52/100 [==============>...............] - ETA: 18s - loss: 0.6658 - accuracy: 0.5933
 53/100 [==============>...............] - ETA: 17s - loss: 0.6669 - accuracy: 0.5906
 54/100 [===============>..............] - ETA: 17s - loss: 0.6667 - accuracy: 0.5917
 55/100 [===============>..............] - ETA: 16s - loss: 0.6659 - accuracy: 0.5936
 56/100 [===============>..............] - ETA: 16s - loss: 0.6660 - accuracy: 0.5911
 57/100 [================>.............] - ETA: 15s - loss: 0.6655 - accuracy: 0.5921
 58/100 [================>.............] - ETA: 15s - loss: 0.6644 - accuracy: 0.5940
 59/100 [================>.............] - ETA: 15s - loss: 0.6642 - accuracy: 0.5958
 60/100 [=================>............] - ETA: 14s - loss: 0.6634 - accuracy: 0.5975
 61/100 [=================>............] - ETA: 14s - loss: 0.6640 - accuracy: 0.5959
 62/100 [=================>............] - ETA: 14s - loss: 0.6633 - accuracy: 0.5976
 63/100 [=================>............] - ETA: 13s - loss: 0.6629 - accuracy: 0.6008
 64/100 [==================>...........] - ETA: 13s - loss: 0.6622 - accuracy: 0.6000
 65/100 [==================>...........] - ETA: 12s - loss: 0.6637 - accuracy: 0.5977
 66/100 [==================>...........] - ETA: 12s - loss: 0.6623 - accuracy: 0.6008
 67/100 [===================>..........] - ETA: 12s - loss: 0.6626 - accuracy: 0.5993
 68/100 [===================>..........] - ETA: 11s - loss: 0.6617 - accuracy: 0.6022
 69/100 [===================>..........] - ETA: 11s - loss: 0.6622 - accuracy: 0.6007
 70/100 [====================>.........] - ETA: 11s - loss: 0.6623 - accuracy: 0.6000
 71/100 [====================>.........] - ETA: 10s - loss: 0.6618 - accuracy: 0.6000
 72/100 [====================>.........] - ETA: 10s - loss: 0.6624 - accuracy: 0.6000
 73/100 [====================>.........] - ETA: 9s - loss: 0.6624 - accuracy: 0.5986 
 74/100 [=====================>........] - ETA: 9s - loss: 0.6622 - accuracy: 0.5986
 75/100 [=====================>........] - ETA: 9s - loss: 0.6606 - accuracy: 0.5993
 76/100 [=====================>........] - ETA: 8s - loss: 0.6593 - accuracy: 0.6020
 77/100 [======================>.......] - ETA: 8s - loss: 0.6593 - accuracy: 0.6026
 78/100 [======================>.......] - ETA: 8s - loss: 0.6594 - accuracy: 0.6032
 79/100 [======================>.......] - ETA: 7s - loss: 0.6594 - accuracy: 0.6038
 80/100 [=======================>......] - ETA: 7s - loss: 0.6593 - accuracy: 0.6050
 81/100 [=======================>......] - ETA: 6s - loss: 0.6589 - accuracy: 0.6056
 82/100 [=======================>......] - ETA: 6s - loss: 0.6575 - accuracy: 0.6067
 83/100 [=======================>......] - ETA: 6s - loss: 0.6577 - accuracy: 0.6054
 84/100 [========================>.....] - ETA: 5s - loss: 0.6570 - accuracy: 0.6060
 85/100 [========================>.....] - ETA: 5s - loss: 0.6571 - accuracy: 0.6059
 86/100 [========================>.....] - ETA: 5s - loss: 0.6570 - accuracy: 0.6058
 87/100 [=========================>....] - ETA: 4s - loss: 0.6579 - accuracy: 0.6052
 88/100 [=========================>....] - ETA: 4s - loss: 0.6575 - accuracy: 0.6074
 89/100 [=========================>....] - ETA: 4s - loss: 0.6565 - accuracy: 0.6079
 90/100 [==========================>...] - ETA: 3s - loss: 0.6559 - accuracy: 0.6089
 91/100 [==========================>...] - ETA: 3s - loss: 0.6550 - accuracy: 0.6099
 92/100 [==========================>...] - ETA: 2s - loss: 0.6548 - accuracy: 0.6098
 93/100 [==========================>...] - ETA: 2s - loss: 0.6552 - accuracy: 0.6081
 94/100 [===========================>..] - ETA: 2s - loss: 0.6557 - accuracy: 0.6074
 95/100 [===========================>..] - ETA: 1s - loss: 0.6556 - accuracy: 0.6079
 96/100 [===========================>..] - ETA: 1s - loss: 0.6547 - accuracy: 0.6094
 97/100 [============================>.] - ETA: 1s - loss: 0.6535 - accuracy: 0.6113
 98/100 [============================>.] - ETA: 0s - loss: 0.6531 - accuracy: 0.6117
 99/100 [============================>.] - ETA: 0s - loss: 0.6536 - accuracy: 0.6106
100/100 [==============================] - 37s 366ms/step - loss: 0.6541 - accuracy: 0.6090

100/100 [==============================] - 41s 410ms/step - loss: 0.6541 - accuracy: 0.6090 - val_loss: 0.6366 - val_accuracy: 0.6370
Epoch 3/30

  1/100 [..............................] - ETA: 28s - loss: 0.7006 - accuracy: 0.5000
  2/100 [..............................] - ETA: 26s - loss: 0.6185 - accuracy: 0.7000
  3/100 [..............................] - ETA: 24s - loss: 0.6190 - accuracy: 0.6500
  4/100 [>.............................] - ETA: 23s - loss: 0.6248 - accuracy: 0.6375
  5/100 [>.............................] - ETA: 23s - loss: 0.6287 - accuracy: 0.6400
  6/100 [>.............................] - ETA: 22s - loss: 0.6329 - accuracy: 0.6333
  7/100 [=>............................] - ETA: 22s - loss: 0.6328 - accuracy: 0.6429
  8/100 [=>............................] - ETA: 21s - loss: 0.6348 - accuracy: 0.6438
  9/100 [=>............................] - ETA: 21s - loss: 0.6387 - accuracy: 0.6500
 10/100 [==>...........................] - ETA: 21s - loss: 0.6277 - accuracy: 0.6600
 11/100 [==>...........................] - ETA: 21s - loss: 0.6332 - accuracy: 0.6364
 12/100 [==>...........................] - ETA: 21s - loss: 0.6302 - accuracy: 0.6500
 13/100 [==>...........................] - ETA: 21s - loss: 0.6381 - accuracy: 0.6385
 14/100 [===>..........................] - ETA: 20s - loss: 0.6418 - accuracy: 0.6286
 15/100 [===>..........................] - ETA: 20s - loss: 0.6399 - accuracy: 0.6267
 16/100 [===>..........................] - ETA: 20s - loss: 0.6385 - accuracy: 0.6250
 17/100 [====>.........................] - ETA: 20s - loss: 0.6414 - accuracy: 0.6235
 18/100 [====>.........................] - ETA: 19s - loss: 0.6382 - accuracy: 0.6278
 19/100 [====>.........................] - ETA: 19s - loss: 0.6396 - accuracy: 0.6289
 20/100 [=====>........................] - ETA: 19s - loss: 0.6366 - accuracy: 0.6275
 21/100 [=====>........................] - ETA: 19s - loss: 0.6373 - accuracy: 0.6262
 22/100 [=====>........................] - ETA: 18s - loss: 0.6346 - accuracy: 0.6295
 23/100 [=====>........................] - ETA: 18s - loss: 0.6303 - accuracy: 0.6370
 24/100 [======>.......................] - ETA: 18s - loss: 0.6268 - accuracy: 0.6438
 25/100 [======>.......................] - ETA: 18s - loss: 0.6294 - accuracy: 0.6420
 26/100 [======>.......................] - ETA: 17s - loss: 0.6280 - accuracy: 0.6442
 27/100 [=======>......................] - ETA: 17s - loss: 0.6273 - accuracy: 0.6426
 28/100 [=======>......................] - ETA: 17s - loss: 0.6281 - accuracy: 0.6393
 29/100 [=======>......................] - ETA: 16s - loss: 0.6301 - accuracy: 0.6362
 30/100 [========>.....................] - ETA: 16s - loss: 0.6292 - accuracy: 0.6350
 31/100 [========>.....................] - ETA: 16s - loss: 0.6254 - accuracy: 0.6419
 32/100 [========>.....................] - ETA: 16s - loss: 0.6251 - accuracy: 0.6406
 33/100 [========>.....................] - ETA: 15s - loss: 0.6239 - accuracy: 0.6439
 34/100 [=========>....................] - ETA: 15s - loss: 0.6219 - accuracy: 0.6485
 35/100 [=========>....................] - ETA: 15s - loss: 0.6229 - accuracy: 0.6471
 36/100 [=========>....................] - ETA: 15s - loss: 0.6232 - accuracy: 0.6486
 37/100 [==========>...................] - ETA: 14s - loss: 0.6200 - accuracy: 0.6554
 38/100 [==========>...................] - ETA: 14s - loss: 0.6208 - accuracy: 0.6526
 39/100 [==========>...................] - ETA: 14s - loss: 0.6216 - accuracy: 0.6474
 40/100 [===========>..................] - ETA: 14s - loss: 0.6248 - accuracy: 0.6450
 41/100 [===========>..................] - ETA: 14s - loss: 0.6250 - accuracy: 0.6451
 42/100 [===========>..................] - ETA: 14s - loss: 0.6256 - accuracy: 0.6464
 43/100 [===========>..................] - ETA: 14s - loss: 0.6246 - accuracy: 0.6477
 44/100 [============>.................] - ETA: 14s - loss: 0.6243 - accuracy: 0.6489
 45/100 [============>.................] - ETA: 14s - loss: 0.6243 - accuracy: 0.6511
 46/100 [============>.................] - ETA: 14s - loss: 0.6234 - accuracy: 0.6511
 47/100 [=============>................] - ETA: 13s - loss: 0.6234 - accuracy: 0.6500
 48/100 [=============>................] - ETA: 13s - loss: 0.6215 - accuracy: 0.6521
 49/100 [=============>................] - ETA: 13s - loss: 0.6236 - accuracy: 0.6469
 50/100 [==============>...............] - ETA: 13s - loss: 0.6192 - accuracy: 0.6520
 51/100 [==============>...............] - ETA: 13s - loss: 0.6198 - accuracy: 0.6520
 52/100 [==============>...............] - ETA: 13s - loss: 0.6176 - accuracy: 0.6538
 53/100 [==============>...............] - ETA: 12s - loss: 0.6186 - accuracy: 0.6538
 54/100 [===============>..............] - ETA: 12s - loss: 0.6165 - accuracy: 0.6574
 55/100 [===============>..............] - ETA: 12s - loss: 0.6154 - accuracy: 0.6582
 56/100 [===============>..............] - ETA: 12s - loss: 0.6154 - accuracy: 0.6598
 57/100 [================>.............] - ETA: 12s - loss: 0.6173 - accuracy: 0.6579
 58/100 [================>.............] - ETA: 11s - loss: 0.6186 - accuracy: 0.6552
 59/100 [================>.............] - ETA: 11s - loss: 0.6205 - accuracy: 0.6542
 60/100 [=================>............] - ETA: 11s - loss: 0.6207 - accuracy: 0.6550
 61/100 [=================>............] - ETA: 11s - loss: 0.6207 - accuracy: 0.6566
 62/100 [=================>............] - ETA: 10s - loss: 0.6189 - accuracy: 0.6581
 63/100 [=================>............] - ETA: 10s - loss: 0.6187 - accuracy: 0.6579
 64/100 [==================>...........] - ETA: 10s - loss: 0.6191 - accuracy: 0.6562
 65/100 [==================>...........] - ETA: 10s - loss: 0.6194 - accuracy: 0.6562
 66/100 [==================>...........] - ETA: 9s - loss: 0.6204 - accuracy: 0.6545 
 67/100 [===================>..........] - ETA: 9s - loss: 0.6196 - accuracy: 0.6560
 68/100 [===================>..........] - ETA: 9s - loss: 0.6215 - accuracy: 0.6529
 69/100 [===================>..........] - ETA: 9s - loss: 0.6198 - accuracy: 0.6543
 70/100 [====================>.........] - ETA: 8s - loss: 0.6204 - accuracy: 0.6529
 71/100 [====================>.........] - ETA: 8s - loss: 0.6202 - accuracy: 0.6528
 72/100 [====================>.........] - ETA: 8s - loss: 0.6206 - accuracy: 0.6514
 73/100 [====================>.........] - ETA: 8s - loss: 0.6214 - accuracy: 0.6521
 74/100 [=====================>........] - ETA: 7s - loss: 0.6204 - accuracy: 0.6547
 75/100 [=====================>........] - ETA: 7s - loss: 0.6201 - accuracy: 0.6553
 76/100 [=====================>........] - ETA: 7s - loss: 0.6214 - accuracy: 0.6539
 77/100 [======================>.......] - ETA: 6s - loss: 0.6212 - accuracy: 0.6545
 78/100 [======================>.......] - ETA: 6s - loss: 0.6208 - accuracy: 0.6538
 79/100 [======================>.......] - ETA: 6s - loss: 0.6219 - accuracy: 0.6525
 80/100 [=======================>......] - ETA: 6s - loss: 0.6212 - accuracy: 0.6538
 81/100 [=======================>......] - ETA: 5s - loss: 0.6207 - accuracy: 0.6537
 82/100 [=======================>......] - ETA: 5s - loss: 0.6201 - accuracy: 0.6549
 83/100 [=======================>......] - ETA: 5s - loss: 0.6197 - accuracy: 0.6542
 84/100 [========================>.....] - ETA: 4s - loss: 0.6193 - accuracy: 0.6548
 85/100 [========================>.....] - ETA: 4s - loss: 0.6195 - accuracy: 0.6547
 86/100 [========================>.....] - ETA: 4s - loss: 0.6190 - accuracy: 0.6547
 87/100 [=========================>....] - ETA: 3s - loss: 0.6198 - accuracy: 0.6523
 88/100 [=========================>....] - ETA: 3s - loss: 0.6182 - accuracy: 0.6545
 89/100 [=========================>....] - ETA: 3s - loss: 0.6193 - accuracy: 0.6539
 90/100 [==========================>...] - ETA: 3s - loss: 0.6187 - accuracy: 0.6550
 91/100 [==========================>...] - ETA: 2s - loss: 0.6189 - accuracy: 0.6544
 92/100 [==========================>...] - ETA: 2s - loss: 0.6180 - accuracy: 0.6543
 93/100 [==========================>...] - ETA: 2s - loss: 0.6182 - accuracy: 0.6527
 94/100 [===========================>..] - ETA: 1s - loss: 0.6178 - accuracy: 0.6532
 95/100 [===========================>..] - ETA: 1s - loss: 0.6182 - accuracy: 0.6532
 96/100 [===========================>..] - ETA: 1s - loss: 0.6181 - accuracy: 0.6531
 97/100 [============================>.] - ETA: 0s - loss: 0.6172 - accuracy: 0.6546
 98/100 [============================>.] - ETA: 0s - loss: 0.6163 - accuracy: 0.6556
 99/100 [============================>.] - ETA: 0s - loss: 0.6161 - accuracy: 0.6561
100/100 [==============================] - 31s 311ms/step - loss: 0.6150 - accuracy: 0.6565

100/100 [==============================] - 35s 353ms/step - loss: 0.6150 - accuracy: 0.6565 - val_loss: 0.6116 - val_accuracy: 0.6640
Epoch 4/30

  1/100 [..............................] - ETA: 25s - loss: 0.5104 - accuracy: 0.7000
  2/100 [..............................] - ETA: 21s - loss: 0.5412 - accuracy: 0.7250
  3/100 [..............................] - ETA: 21s - loss: 0.5889 - accuracy: 0.6833
  4/100 [>.............................] - ETA: 20s - loss: 0.5814 - accuracy: 0.7125
  5/100 [>.............................] - ETA: 20s - loss: 0.5703 - accuracy: 0.7300
  6/100 [>.............................] - ETA: 20s - loss: 0.5690 - accuracy: 0.7250
  7/100 [=>............................] - ETA: 19s - loss: 0.5544 - accuracy: 0.7429
  8/100 [=>............................] - ETA: 19s - loss: 0.5507 - accuracy: 0.7437
  9/100 [=>............................] - ETA: 19s - loss: 0.5401 - accuracy: 0.7500
 10/100 [==>...........................] - ETA: 19s - loss: 0.5389 - accuracy: 0.7550
 11/100 [==>...........................] - ETA: 19s - loss: 0.5485 - accuracy: 0.7455
 12/100 [==>...........................] - ETA: 18s - loss: 0.5544 - accuracy: 0.7250
 13/100 [==>...........................] - ETA: 18s - loss: 0.5642 - accuracy: 0.7115
 14/100 [===>..........................] - ETA: 18s - loss: 0.5704 - accuracy: 0.7143
 15/100 [===>..........................] - ETA: 18s - loss: 0.5693 - accuracy: 0.7133
 16/100 [===>..........................] - ETA: 18s - loss: 0.5731 - accuracy: 0.7063
 17/100 [====>.........................] - ETA: 17s - loss: 0.5709 - accuracy: 0.7118
 18/100 [====>.........................] - ETA: 17s - loss: 0.5726 - accuracy: 0.7111
 19/100 [====>.........................] - ETA: 17s - loss: 0.5691 - accuracy: 0.7105
 20/100 [=====>........................] - ETA: 17s - loss: 0.5645 - accuracy: 0.7125
 21/100 [=====>........................] - ETA: 17s - loss: 0.5647 - accuracy: 0.7095
 22/100 [=====>........................] - ETA: 17s - loss: 0.5611 - accuracy: 0.7136
 23/100 [=====>........................] - ETA: 18s - loss: 0.5604 - accuracy: 0.7109
 24/100 [======>.......................] - ETA: 18s - loss: 0.5592 - accuracy: 0.7125
 25/100 [======>.......................] - ETA: 18s - loss: 0.5611 - accuracy: 0.7100
 26/100 [======>.......................] - ETA: 18s - loss: 0.5587 - accuracy: 0.7115
 27/100 [=======>......................] - ETA: 18s - loss: 0.5580 - accuracy: 0.7130
 28/100 [=======>......................] - ETA: 18s - loss: 0.5606 - accuracy: 0.7089
 29/100 [=======>......................] - ETA: 18s - loss: 0.5644 - accuracy: 0.7017
 30/100 [========>.....................] - ETA: 18s - loss: 0.5655 - accuracy: 0.7017
 31/100 [========>.....................] - ETA: 18s - loss: 0.5644 - accuracy: 0.7048
 32/100 [========>.....................] - ETA: 18s - loss: 0.5642 - accuracy: 0.7047
 33/100 [========>.....................] - ETA: 18s - loss: 0.5633 - accuracy: 0.7076
 34/100 [=========>....................] - ETA: 18s - loss: 0.5608 - accuracy: 0.7088
 35/100 [=========>....................] - ETA: 17s - loss: 0.5656 - accuracy: 0.7014
 36/100 [=========>....................] - ETA: 17s - loss: 0.5703 - accuracy: 0.6958
 37/100 [==========>...................] - ETA: 17s - loss: 0.5681 - accuracy: 0.6973
 38/100 [==========>...................] - ETA: 17s - loss: 0.5687 - accuracy: 0.6974
 39/100 [==========>...................] - ETA: 17s - loss: 0.5693 - accuracy: 0.6962
 40/100 [===========>..................] - ETA: 17s - loss: 0.5673 - accuracy: 0.6975
 41/100 [===========>..................] - ETA: 16s - loss: 0.5647 - accuracy: 0.7000
 42/100 [===========>..................] - ETA: 16s - loss: 0.5626 - accuracy: 0.7024
 43/100 [===========>..................] - ETA: 16s - loss: 0.5636 - accuracy: 0.7012
 44/100 [============>.................] - ETA: 16s - loss: 0.5630 - accuracy: 0.7023
 45/100 [============>.................] - ETA: 15s - loss: 0.5619 - accuracy: 0.7044
 46/100 [============>.................] - ETA: 15s - loss: 0.5622 - accuracy: 0.7043
 47/100 [=============>................] - ETA: 15s - loss: 0.5640 - accuracy: 0.7021
 48/100 [=============>................] - ETA: 15s - loss: 0.5644 - accuracy: 0.7021
 49/100 [=============>................] - ETA: 15s - loss: 0.5659 - accuracy: 0.7010
 50/100 [==============>...............] - ETA: 14s - loss: 0.5633 - accuracy: 0.7020
 51/100 [==============>...............] - ETA: 14s - loss: 0.5657 - accuracy: 0.7010
 52/100 [==============>...............] - ETA: 14s - loss: 0.5653 - accuracy: 0.7010
 53/100 [==============>...............] - ETA: 14s - loss: 0.5664 - accuracy: 0.7000
 54/100 [===============>..............] - ETA: 13s - loss: 0.5643 - accuracy: 0.7019
 55/100 [===============>..............] - ETA: 13s - loss: 0.5632 - accuracy: 0.7027
 56/100 [===============>..............] - ETA: 13s - loss: 0.5642 - accuracy: 0.7018
 57/100 [================>.............] - ETA: 13s - loss: 0.5659 - accuracy: 0.7009
 58/100 [================>.............] - ETA: 12s - loss: 0.5640 - accuracy: 0.7026
 59/100 [================>.............] - ETA: 12s - loss: 0.5646 - accuracy: 0.7008
 60/100 [=================>............] - ETA: 12s - loss: 0.5644 - accuracy: 0.7008
 61/100 [=================>............] - ETA: 11s - loss: 0.5660 - accuracy: 0.6984
 62/100 [=================>............] - ETA: 11s - loss: 0.5667 - accuracy: 0.7000
 63/100 [=================>............] - ETA: 11s - loss: 0.5648 - accuracy: 0.7024
 64/100 [==================>...........] - ETA: 11s - loss: 0.5653 - accuracy: 0.7031
 65/100 [==================>...........] - ETA: 10s - loss: 0.5677 - accuracy: 0.6992
 66/100 [==================>...........] - ETA: 10s - loss: 0.5705 - accuracy: 0.6947
 67/100 [===================>..........] - ETA: 10s - loss: 0.5708 - accuracy: 0.6955
 68/100 [===================>..........] - ETA: 9s - loss: 0.5711 - accuracy: 0.6956 
 69/100 [===================>..........] - ETA: 9s - loss: 0.5696 - accuracy: 0.6971
 70/100 [====================>.........] - ETA: 9s - loss: 0.5689 - accuracy: 0.6986
 71/100 [====================>.........] - ETA: 9s - loss: 0.5675 - accuracy: 0.7007
 72/100 [====================>.........] - ETA: 8s - loss: 0.5677 - accuracy: 0.7000
 73/100 [====================>.........] - ETA: 8s - loss: 0.5691 - accuracy: 0.6993
 74/100 [=====================>........] - ETA: 8s - loss: 0.5703 - accuracy: 0.6986
 75/100 [=====================>........] - ETA: 7s - loss: 0.5706 - accuracy: 0.6973
 76/100 [=====================>........] - ETA: 7s - loss: 0.5702 - accuracy: 0.6974
 77/100 [======================>.......] - ETA: 7s - loss: 0.5690 - accuracy: 0.6987
 78/100 [======================>.......] - ETA: 6s - loss: 0.5685 - accuracy: 0.6994
 79/100 [======================>.......] - ETA: 6s - loss: 0.5669 - accuracy: 0.7013
 80/100 [=======================>......] - ETA: 6s - loss: 0.5661 - accuracy: 0.7019
 81/100 [=======================>......] - ETA: 6s - loss: 0.5645 - accuracy: 0.7043
 82/100 [=======================>......] - ETA: 5s - loss: 0.5635 - accuracy: 0.7049
 83/100 [=======================>......] - ETA: 5s - loss: 0.5629 - accuracy: 0.7060
 84/100 [========================>.....] - ETA: 5s - loss: 0.5643 - accuracy: 0.7036
 85/100 [========================>.....] - ETA: 4s - loss: 0.5653 - accuracy: 0.7018
 86/100 [========================>.....] - ETA: 4s - loss: 0.5648 - accuracy: 0.7017
 87/100 [=========================>....] - ETA: 4s - loss: 0.5636 - accuracy: 0.7029
 88/100 [=========================>....] - ETA: 3s - loss: 0.5646 - accuracy: 0.7023
 89/100 [=========================>....] - ETA: 3s - loss: 0.5667 - accuracy: 0.7017
 90/100 [==========================>...] - ETA: 3s - loss: 0.5688 - accuracy: 0.7006
 91/100 [==========================>...] - ETA: 2s - loss: 0.5713 - accuracy: 0.6989
 92/100 [==========================>...] - ETA: 2s - loss: 0.5713 - accuracy: 0.6989
 93/100 [==========================>...] - ETA: 2s - loss: 0.5718 - accuracy: 0.6973
 94/100 [===========================>..] - ETA: 1s - loss: 0.5705 - accuracy: 0.6995
 95/100 [===========================>..] - ETA: 1s - loss: 0.5707 - accuracy: 0.6989
 96/100 [===========================>..] - ETA: 1s - loss: 0.5718 - accuracy: 0.6979
 97/100 [============================>.] - ETA: 0s - loss: 0.5731 - accuracy: 0.6959
 98/100 [============================>.] - ETA: 0s - loss: 0.5723 - accuracy: 0.6969
 99/100 [============================>.] - ETA: 0s - loss: 0.5727 - accuracy: 0.6975
100/100 [==============================] - 32s 325ms/step - loss: 0.5731 - accuracy: 0.6970

100/100 [==============================] - 37s 366ms/step - loss: 0.5731 - accuracy: 0.6970 - val_loss: 0.5952 - val_accuracy: 0.6680
Epoch 5/30

  1/100 [..............................] - ETA: 27s - loss: 0.5577 - accuracy: 0.6000
  2/100 [..............................] - ETA: 22s - loss: 0.5869 - accuracy: 0.6250
  3/100 [..............................] - ETA: 23s - loss: 0.5542 - accuracy: 0.7167
  4/100 [>.............................] - ETA: 22s - loss: 0.5590 - accuracy: 0.7250
  5/100 [>.............................] - ETA: 22s - loss: 0.5626 - accuracy: 0.7200
  6/100 [>.............................] - ETA: 21s - loss: 0.5574 - accuracy: 0.7250
  7/100 [=>............................] - ETA: 21s - loss: 0.5565 - accuracy: 0.7429
  8/100 [=>............................] - ETA: 21s - loss: 0.5528 - accuracy: 0.7437
  9/100 [=>............................] - ETA: 21s - loss: 0.5619 - accuracy: 0.7278
 10/100 [==>...........................] - ETA: 21s - loss: 0.5539 - accuracy: 0.7300
 11/100 [==>...........................] - ETA: 20s - loss: 0.5498 - accuracy: 0.7364
 12/100 [==>...........................] - ETA: 20s - loss: 0.5398 - accuracy: 0.7458
 13/100 [==>...........................] - ETA: 20s - loss: 0.5368 - accuracy: 0.7500
 14/100 [===>..........................] - ETA: 19s - loss: 0.5340 - accuracy: 0.7571
 15/100 [===>..........................] - ETA: 19s - loss: 0.5334 - accuracy: 0.7600
 16/100 [===>..........................] - ETA: 19s - loss: 0.5422 - accuracy: 0.7437
 17/100 [====>.........................] - ETA: 18s - loss: 0.5465 - accuracy: 0.7353
 18/100 [====>.........................] - ETA: 18s - loss: 0.5516 - accuracy: 0.7278
 19/100 [====>.........................] - ETA: 18s - loss: 0.5552 - accuracy: 0.7289
 20/100 [=====>........................] - ETA: 18s - loss: 0.5589 - accuracy: 0.7250
 21/100 [=====>........................] - ETA: 17s - loss: 0.5630 - accuracy: 0.7238
 22/100 [=====>........................] - ETA: 17s - loss: 0.5578 - accuracy: 0.7295
 23/100 [=====>........................] - ETA: 17s - loss: 0.5566 - accuracy: 0.7348
 24/100 [======>.......................] - ETA: 17s - loss: 0.5579 - accuracy: 0.7333
 25/100 [======>.......................] - ETA: 16s - loss: 0.5587 - accuracy: 0.7300
 26/100 [======>.......................] - ETA: 16s - loss: 0.5618 - accuracy: 0.7231
 27/100 [=======>......................] - ETA: 16s - loss: 0.5594 - accuracy: 0.7259
 28/100 [=======>......................] - ETA: 17s - loss: 0.5583 - accuracy: 0.7232
 29/100 [=======>......................] - ETA: 17s - loss: 0.5583 - accuracy: 0.7241
 30/100 [========>.....................] - ETA: 17s - loss: 0.5642 - accuracy: 0.7150
 31/100 [========>.....................] - ETA: 17s - loss: 0.5657 - accuracy: 0.7129
 32/100 [========>.....................] - ETA: 17s - loss: 0.5685 - accuracy: 0.7109
 33/100 [========>.....................] - ETA: 17s - loss: 0.5679 - accuracy: 0.7136
 34/100 [=========>....................] - ETA: 17s - loss: 0.5692 - accuracy: 0.7118
 35/100 [=========>....................] - ETA: 16s - loss: 0.5672 - accuracy: 0.7157
 36/100 [=========>....................] - ETA: 16s - loss: 0.5685 - accuracy: 0.7125
 37/100 [==========>...................] - ETA: 16s - loss: 0.5678 - accuracy: 0.7122
 38/100 [==========>...................] - ETA: 16s - loss: 0.5662 - accuracy: 0.7145
 39/100 [==========>...................] - ETA: 16s - loss: 0.5629 - accuracy: 0.7167
 40/100 [===========>..................] - ETA: 16s - loss: 0.5622 - accuracy: 0.7163
 41/100 [===========>..................] - ETA: 16s - loss: 0.5592 - accuracy: 0.7195
 42/100 [===========>..................] - ETA: 15s - loss: 0.5586 - accuracy: 0.7167
 43/100 [===========>..................] - ETA: 15s - loss: 0.5545 - accuracy: 0.7198
 44/100 [============>.................] - ETA: 15s - loss: 0.5566 - accuracy: 0.7193
 45/100 [============>.................] - ETA: 15s - loss: 0.5555 - accuracy: 0.7189
 46/100 [============>.................] - ETA: 15s - loss: 0.5572 - accuracy: 0.7207
 47/100 [=============>................] - ETA: 14s - loss: 0.5570 - accuracy: 0.7191
 48/100 [=============>................] - ETA: 14s - loss: 0.5565 - accuracy: 0.7177
 49/100 [=============>................] - ETA: 14s - loss: 0.5554 - accuracy: 0.7184
 50/100 [==============>...............] - ETA: 14s - loss: 0.5550 - accuracy: 0.7180
 51/100 [==============>...............] - ETA: 14s - loss: 0.5524 - accuracy: 0.7206
 52/100 [==============>...............] - ETA: 13s - loss: 0.5545 - accuracy: 0.7173
 53/100 [==============>...............] - ETA: 13s - loss: 0.5537 - accuracy: 0.7189
 54/100 [===============>..............] - ETA: 13s - loss: 0.5536 - accuracy: 0.7176
 55/100 [===============>..............] - ETA: 13s - loss: 0.5515 - accuracy: 0.7209
 56/100 [===============>..............] - ETA: 13s - loss: 0.5497 - accuracy: 0.7223
 57/100 [================>.............] - ETA: 12s - loss: 0.5520 - accuracy: 0.7202
 58/100 [================>.............] - ETA: 12s - loss: 0.5488 - accuracy: 0.7224
 59/100 [================>.............] - ETA: 12s - loss: 0.5492 - accuracy: 0.7220
 60/100 [=================>............] - ETA: 11s - loss: 0.5516 - accuracy: 0.7183
 61/100 [=================>............] - ETA: 11s - loss: 0.5509 - accuracy: 0.7197
 62/100 [=================>............] - ETA: 11s - loss: 0.5517 - accuracy: 0.7185
 63/100 [=================>............] - ETA: 11s - loss: 0.5506 - accuracy: 0.7190
 64/100 [==================>...........] - ETA: 10s - loss: 0.5481 - accuracy: 0.7227
 65/100 [==================>...........] - ETA: 10s - loss: 0.5477 - accuracy: 0.7238
 66/100 [==================>...........] - ETA: 10s - loss: 0.5528 - accuracy: 0.7197
 67/100 [===================>..........] - ETA: 9s - loss: 0.5528 - accuracy: 0.7209 
 68/100 [===================>..........] - ETA: 9s - loss: 0.5513 - accuracy: 0.7221
 69/100 [===================>..........] - ETA: 9s - loss: 0.5489 - accuracy: 0.7239
 70/100 [====================>.........] - ETA: 9s - loss: 0.5502 - accuracy: 0.7229
 71/100 [====================>.........] - ETA: 8s - loss: 0.5493 - accuracy: 0.7232
 72/100 [====================>.........] - ETA: 8s - loss: 0.5489 - accuracy: 0.7236
 73/100 [====================>.........] - ETA: 8s - loss: 0.5479 - accuracy: 0.7247
 74/100 [=====================>........] - ETA: 8s - loss: 0.5455 - accuracy: 0.7264
 75/100 [=====================>........] - ETA: 7s - loss: 0.5452 - accuracy: 0.7267
 76/100 [=====================>........] - ETA: 7s - loss: 0.5445 - accuracy: 0.7263
 77/100 [======================>.......] - ETA: 7s - loss: 0.5434 - accuracy: 0.7279
 78/100 [======================>.......] - ETA: 6s - loss: 0.5431 - accuracy: 0.7288
 79/100 [======================>.......] - ETA: 6s - loss: 0.5436 - accuracy: 0.7291
 80/100 [=======================>......] - ETA: 6s - loss: 0.5437 - accuracy: 0.7306
 81/100 [=======================>......] - ETA: 5s - loss: 0.5432 - accuracy: 0.7302
 82/100 [=======================>......] - ETA: 5s - loss: 0.5435 - accuracy: 0.7299
 83/100 [=======================>......] - ETA: 5s - loss: 0.5438 - accuracy: 0.7271
 84/100 [========================>.....] - ETA: 5s - loss: 0.5448 - accuracy: 0.7250
 85/100 [========================>.....] - ETA: 4s - loss: 0.5444 - accuracy: 0.7253
 86/100 [========================>.....] - ETA: 4s - loss: 0.5441 - accuracy: 0.7262
 87/100 [=========================>....] - ETA: 4s - loss: 0.5446 - accuracy: 0.7253
 88/100 [=========================>....] - ETA: 3s - loss: 0.5437 - accuracy: 0.7256
 89/100 [=========================>....] - ETA: 3s - loss: 0.5426 - accuracy: 0.7258
 90/100 [==========================>...] - ETA: 3s - loss: 0.5417 - accuracy: 0.7261
 91/100 [==========================>...] - ETA: 2s - loss: 0.5408 - accuracy: 0.7280
 92/100 [==========================>...] - ETA: 2s - loss: 0.5400 - accuracy: 0.7283
 93/100 [==========================>...] - ETA: 2s - loss: 0.5396 - accuracy: 0.7290
 94/100 [===========================>..] - ETA: 1s - loss: 0.5395 - accuracy: 0.7287
 95/100 [===========================>..] - ETA: 1s - loss: 0.5384 - accuracy: 0.7300
 96/100 [===========================>..] - ETA: 1s - loss: 0.5386 - accuracy: 0.7302
 97/100 [============================>.] - ETA: 0s - loss: 0.5374 - accuracy: 0.7304
 98/100 [============================>.] - ETA: 0s - loss: 0.5391 - accuracy: 0.7286
 99/100 [============================>.] - ETA: 0s - loss: 0.5398 - accuracy: 0.7283
100/100 [==============================] - 32s 319ms/step - loss: 0.5386 - accuracy: 0.7290

100/100 [==============================] - 36s 362ms/step - loss: 0.5386 - accuracy: 0.7290 - val_loss: 0.7160 - val_accuracy: 0.6090
Epoch 6/30

  1/100 [..............................] - ETA: 27s - loss: 0.5573 - accuracy: 0.6500
  2/100 [..............................] - ETA: 20s - loss: 0.5557 - accuracy: 0.6500
  3/100 [..............................] - ETA: 20s - loss: 0.5232 - accuracy: 0.6667
  4/100 [>.............................] - ETA: 20s - loss: 0.5520 - accuracy: 0.6750
  5/100 [>.............................] - ETA: 20s - loss: 0.5339 - accuracy: 0.7000
  6/100 [>.............................] - ETA: 20s - loss: 0.5110 - accuracy: 0.7417
  7/100 [=>............................] - ETA: 20s - loss: 0.5173 - accuracy: 0.7286
  8/100 [=>............................] - ETA: 20s - loss: 0.5083 - accuracy: 0.7250
  9/100 [=>............................] - ETA: 19s - loss: 0.5102 - accuracy: 0.7222
 10/100 [==>...........................] - ETA: 19s - loss: 0.5012 - accuracy: 0.7300
 11/100 [==>...........................] - ETA: 19s - loss: 0.5125 - accuracy: 0.7273
 12/100 [==>...........................] - ETA: 19s - loss: 0.5146 - accuracy: 0.7208
 13/100 [==>...........................] - ETA: 19s - loss: 0.5096 - accuracy: 0.7269
 14/100 [===>..........................] - ETA: 18s - loss: 0.4976 - accuracy: 0.7357
 15/100 [===>..........................] - ETA: 18s - loss: 0.4949 - accuracy: 0.7433
 16/100 [===>..........................] - ETA: 18s - loss: 0.5030 - accuracy: 0.7469
 17/100 [====>.........................] - ETA: 18s - loss: 0.5069 - accuracy: 0.7382
 18/100 [====>.........................] - ETA: 17s - loss: 0.5019 - accuracy: 0.7417
 19/100 [====>.........................] - ETA: 17s - loss: 0.4987 - accuracy: 0.7421
 20/100 [=====>........................] - ETA: 17s - loss: 0.5080 - accuracy: 0.7375
 21/100 [=====>........................] - ETA: 17s - loss: 0.5048 - accuracy: 0.7405
 22/100 [=====>........................] - ETA: 16s - loss: 0.5057 - accuracy: 0.7409
 23/100 [=====>........................] - ETA: 16s - loss: 0.5064 - accuracy: 0.7435
 24/100 [======>.......................] - ETA: 16s - loss: 0.5123 - accuracy: 0.7396
 25/100 [======>.......................] - ETA: 16s - loss: 0.5066 - accuracy: 0.7480
 26/100 [======>.......................] - ETA: 17s - loss: 0.5083 - accuracy: 0.7462
 27/100 [=======>......................] - ETA: 17s - loss: 0.5097 - accuracy: 0.7481
 28/100 [=======>......................] - ETA: 17s - loss: 0.5032 - accuracy: 0.7536
 29/100 [=======>......................] - ETA: 17s - loss: 0.5008 - accuracy: 0.7534
 30/100 [========>.....................] - ETA: 17s - loss: 0.4935 - accuracy: 0.7600
 31/100 [========>.....................] - ETA: 17s - loss: 0.4920 - accuracy: 0.7629
 32/100 [========>.....................] - ETA: 17s - loss: 0.4998 - accuracy: 0.7547
 33/100 [========>.....................] - ETA: 17s - loss: 0.5079 - accuracy: 0.7500
 34/100 [=========>....................] - ETA: 17s - loss: 0.5086 - accuracy: 0.7485
 35/100 [=========>....................] - ETA: 17s - loss: 0.5138 - accuracy: 0.7443
 36/100 [=========>....................] - ETA: 17s - loss: 0.5129 - accuracy: 0.7444
 37/100 [==========>...................] - ETA: 17s - loss: 0.5126 - accuracy: 0.7419
 38/100 [==========>...................] - ETA: 17s - loss: 0.5127 - accuracy: 0.7434
 39/100 [==========>...................] - ETA: 17s - loss: 0.5108 - accuracy: 0.7462
 40/100 [===========>..................] - ETA: 16s - loss: 0.5095 - accuracy: 0.7487
 41/100 [===========>..................] - ETA: 16s - loss: 0.5082 - accuracy: 0.7524
 42/100 [===========>..................] - ETA: 16s - loss: 0.5064 - accuracy: 0.7560
 43/100 [===========>..................] - ETA: 16s - loss: 0.5040 - accuracy: 0.7558
 44/100 [============>.................] - ETA: 16s - loss: 0.5050 - accuracy: 0.7545
 45/100 [============>.................] - ETA: 15s - loss: 0.5085 - accuracy: 0.7511
 46/100 [============>.................] - ETA: 15s - loss: 0.5094 - accuracy: 0.7489
 47/100 [=============>................] - ETA: 15s - loss: 0.5087 - accuracy: 0.7479
 48/100 [=============>................] - ETA: 15s - loss: 0.5109 - accuracy: 0.7469
 49/100 [=============>................] - ETA: 14s - loss: 0.5098 - accuracy: 0.7480
 50/100 [==============>...............] - ETA: 14s - loss: 0.5083 - accuracy: 0.7510
 51/100 [==============>...............] - ETA: 14s - loss: 0.5146 - accuracy: 0.7471
 52/100 [==============>...............] - ETA: 14s - loss: 0.5156 - accuracy: 0.7452
 53/100 [==============>...............] - ETA: 13s - loss: 0.5169 - accuracy: 0.7453
 54/100 [===============>..............] - ETA: 13s - loss: 0.5158 - accuracy: 0.7463
 55/100 [===============>..............] - ETA: 13s - loss: 0.5144 - accuracy: 0.7482
 56/100 [===============>..............] - ETA: 13s - loss: 0.5129 - accuracy: 0.7500
 57/100 [================>.............] - ETA: 12s - loss: 0.5106 - accuracy: 0.7500
 58/100 [================>.............] - ETA: 12s - loss: 0.5135 - accuracy: 0.7457
 59/100 [================>.............] - ETA: 12s - loss: 0.5128 - accuracy: 0.7475
 60/100 [=================>............] - ETA: 11s - loss: 0.5106 - accuracy: 0.7492
 61/100 [=================>............] - ETA: 11s - loss: 0.5101 - accuracy: 0.7508
 62/100 [=================>............] - ETA: 11s - loss: 0.5091 - accuracy: 0.7516
 63/100 [=================>............] - ETA: 11s - loss: 0.5097 - accuracy: 0.7516
 64/100 [==================>...........] - ETA: 10s - loss: 0.5100 - accuracy: 0.7508
 65/100 [==================>...........] - ETA: 10s - loss: 0.5093 - accuracy: 0.7500
 66/100 [==================>...........] - ETA: 10s - loss: 0.5095 - accuracy: 0.7508
 67/100 [===================>..........] - ETA: 10s - loss: 0.5083 - accuracy: 0.7530
 68/100 [===================>..........] - ETA: 9s - loss: 0.5076 - accuracy: 0.7537 
 69/100 [===================>..........] - ETA: 9s - loss: 0.5055 - accuracy: 0.7565
 70/100 [====================>.........] - ETA: 9s - loss: 0.5058 - accuracy: 0.7571
 71/100 [====================>.........] - ETA: 8s - loss: 0.5052 - accuracy: 0.7585
 72/100 [====================>.........] - ETA: 8s - loss: 0.5048 - accuracy: 0.7583
 73/100 [====================>.........] - ETA: 8s - loss: 0.5058 - accuracy: 0.7575
 74/100 [=====================>........] - ETA: 8s - loss: 0.5066 - accuracy: 0.7581
 75/100 [=====================>........] - ETA: 7s - loss: 0.5094 - accuracy: 0.7547
 76/100 [=====================>........] - ETA: 7s - loss: 0.5106 - accuracy: 0.7546
 77/100 [======================>.......] - ETA: 7s - loss: 0.5104 - accuracy: 0.7539
 78/100 [======================>.......] - ETA: 6s - loss: 0.5128 - accuracy: 0.7519
 79/100 [======================>.......] - ETA: 6s - loss: 0.5111 - accuracy: 0.7532
 80/100 [=======================>......] - ETA: 6s - loss: 0.5107 - accuracy: 0.7538
 81/100 [=======================>......] - ETA: 5s - loss: 0.5093 - accuracy: 0.7562
 82/100 [=======================>......] - ETA: 5s - loss: 0.5101 - accuracy: 0.7543
 83/100 [=======================>......] - ETA: 5s - loss: 0.5113 - accuracy: 0.7536
 84/100 [========================>.....] - ETA: 5s - loss: 0.5107 - accuracy: 0.7548
 85/100 [========================>.....] - ETA: 4s - loss: 0.5088 - accuracy: 0.7559
 86/100 [========================>.....] - ETA: 4s - loss: 0.5123 - accuracy: 0.7529
 87/100 [=========================>....] - ETA: 4s - loss: 0.5163 - accuracy: 0.7506
 88/100 [=========================>....] - ETA: 3s - loss: 0.5156 - accuracy: 0.7517
 89/100 [=========================>....] - ETA: 3s - loss: 0.5151 - accuracy: 0.7522
 90/100 [==========================>...] - ETA: 3s - loss: 0.5146 - accuracy: 0.7528
 91/100 [==========================>...] - ETA: 2s - loss: 0.5138 - accuracy: 0.7533
 92/100 [==========================>...] - ETA: 2s - loss: 0.5145 - accuracy: 0.7533
 93/100 [==========================>...] - ETA: 2s - loss: 0.5134 - accuracy: 0.7543
 94/100 [===========================>..] - ETA: 1s - loss: 0.5124 - accuracy: 0.7559
 95/100 [===========================>..] - ETA: 1s - loss: 0.5125 - accuracy: 0.7558
 96/100 [===========================>..] - ETA: 1s - loss: 0.5128 - accuracy: 0.7557
 97/100 [============================>.] - ETA: 0s - loss: 0.5123 - accuracy: 0.7557
 98/100 [============================>.] - ETA: 0s - loss: 0.5112 - accuracy: 0.7561
 99/100 [============================>.] - ETA: 0s - loss: 0.5119 - accuracy: 0.7556
100/100 [==============================] - 32s 319ms/step - loss: 0.5110 - accuracy: 0.7555

100/100 [==============================] - 36s 362ms/step - loss: 0.5110 - accuracy: 0.7555 - val_loss: 0.5651 - val_accuracy: 0.7000
Epoch 7/30

  1/100 [..............................] - ETA: 27s - loss: 0.3974 - accuracy: 0.9000
  2/100 [..............................] - ETA: 21s - loss: 0.4797 - accuracy: 0.8500
  3/100 [..............................] - ETA: 20s - loss: 0.4375 - accuracy: 0.8667
  4/100 [>.............................] - ETA: 21s - loss: 0.4879 - accuracy: 0.8000
  5/100 [>.............................] - ETA: 20s - loss: 0.4993 - accuracy: 0.7900
  6/100 [>.............................] - ETA: 20s - loss: 0.4949 - accuracy: 0.8000
  7/100 [=>............................] - ETA: 20s - loss: 0.5060 - accuracy: 0.7929
  8/100 [=>............................] - ETA: 20s - loss: 0.5140 - accuracy: 0.7812
  9/100 [=>............................] - ETA: 20s - loss: 0.5141 - accuracy: 0.7722
 10/100 [==>...........................] - ETA: 20s - loss: 0.5098 - accuracy: 0.7700
 11/100 [==>...........................] - ETA: 20s - loss: 0.5018 - accuracy: 0.7682
 12/100 [==>...........................] - ETA: 19s - loss: 0.4914 - accuracy: 0.7708
 13/100 [==>...........................] - ETA: 19s - loss: 0.4961 - accuracy: 0.7692
 14/100 [===>..........................] - ETA: 19s - loss: 0.4950 - accuracy: 0.7643
 15/100 [===>..........................] - ETA: 19s - loss: 0.4930 - accuracy: 0.7700
 16/100 [===>..........................] - ETA: 18s - loss: 0.4951 - accuracy: 0.7688
 17/100 [====>.........................] - ETA: 18s - loss: 0.4977 - accuracy: 0.7676
 18/100 [====>.........................] - ETA: 18s - loss: 0.4951 - accuracy: 0.7694
 19/100 [====>.........................] - ETA: 18s - loss: 0.4890 - accuracy: 0.7737
 20/100 [=====>........................] - ETA: 17s - loss: 0.4830 - accuracy: 0.7800
 21/100 [=====>........................] - ETA: 17s - loss: 0.4820 - accuracy: 0.7762
 22/100 [=====>........................] - ETA: 17s - loss: 0.4802 - accuracy: 0.7727
 23/100 [=====>........................] - ETA: 17s - loss: 0.4792 - accuracy: 0.7739
 24/100 [======>.......................] - ETA: 16s - loss: 0.4794 - accuracy: 0.7750
 25/100 [======>.......................] - ETA: 16s - loss: 0.4812 - accuracy: 0.7740
 26/100 [======>.......................] - ETA: 16s - loss: 0.4787 - accuracy: 0.7731
 27/100 [=======>......................] - ETA: 16s - loss: 0.4772 - accuracy: 0.7741
 28/100 [=======>......................] - ETA: 16s - loss: 0.4762 - accuracy: 0.7714
 29/100 [=======>......................] - ETA: 16s - loss: 0.4736 - accuracy: 0.7741
 30/100 [========>.....................] - ETA: 17s - loss: 0.4695 - accuracy: 0.7800
 31/100 [========>.....................] - ETA: 16s - loss: 0.4692 - accuracy: 0.7823
 32/100 [========>.....................] - ETA: 17s - loss: 0.4723 - accuracy: 0.7797
 33/100 [========>.....................] - ETA: 16s - loss: 0.4761 - accuracy: 0.7758
 34/100 [=========>....................] - ETA: 16s - loss: 0.4781 - accuracy: 0.7735
 35/100 [=========>....................] - ETA: 16s - loss: 0.4815 - accuracy: 0.7714
 36/100 [=========>....................] - ETA: 16s - loss: 0.4798 - accuracy: 0.7736
 37/100 [==========>...................] - ETA: 16s - loss: 0.4788 - accuracy: 0.7743
 38/100 [==========>...................] - ETA: 16s - loss: 0.4767 - accuracy: 0.7763
 39/100 [==========>...................] - ETA: 16s - loss: 0.4773 - accuracy: 0.7769
 40/100 [===========>..................] - ETA: 16s - loss: 0.4731 - accuracy: 0.7800
 41/100 [===========>..................] - ETA: 16s - loss: 0.4730 - accuracy: 0.7793
 42/100 [===========>..................] - ETA: 16s - loss: 0.4701 - accuracy: 0.7810
 43/100 [===========>..................] - ETA: 15s - loss: 0.4696 - accuracy: 0.7826
 44/100 [============>.................] - ETA: 15s - loss: 0.4708 - accuracy: 0.7818
 45/100 [============>.................] - ETA: 15s - loss: 0.4721 - accuracy: 0.7811
 46/100 [============>.................] - ETA: 15s - loss: 0.4771 - accuracy: 0.7772
 47/100 [=============>................] - ETA: 14s - loss: 0.4757 - accuracy: 0.7766
 48/100 [=============>................] - ETA: 14s - loss: 0.4776 - accuracy: 0.7750
 49/100 [=============>................] - ETA: 14s - loss: 0.4819 - accuracy: 0.7745
 50/100 [==============>...............] - ETA: 14s - loss: 0.4816 - accuracy: 0.7740
 51/100 [==============>...............] - ETA: 14s - loss: 0.4804 - accuracy: 0.7745
 52/100 [==============>...............] - ETA: 13s - loss: 0.4821 - accuracy: 0.7740
 53/100 [==============>...............] - ETA: 13s - loss: 0.4809 - accuracy: 0.7764
 54/100 [===============>..............] - ETA: 13s - loss: 0.4799 - accuracy: 0.7787
 55/100 [===============>..............] - ETA: 13s - loss: 0.4771 - accuracy: 0.7809
 56/100 [===============>..............] - ETA: 12s - loss: 0.4757 - accuracy: 0.7821
 57/100 [================>.............] - ETA: 12s - loss: 0.4775 - accuracy: 0.7807
 58/100 [================>.............] - ETA: 12s - loss: 0.4841 - accuracy: 0.7759
 59/100 [================>.............] - ETA: 12s - loss: 0.4842 - accuracy: 0.7746
 60/100 [=================>............] - ETA: 11s - loss: 0.4865 - accuracy: 0.7733
 61/100 [=================>............] - ETA: 11s - loss: 0.4861 - accuracy: 0.7738
 62/100 [=================>............] - ETA: 11s - loss: 0.4854 - accuracy: 0.7766
 63/100 [=================>............] - ETA: 11s - loss: 0.4871 - accuracy: 0.7754
 64/100 [==================>...........] - ETA: 10s - loss: 0.4837 - accuracy: 0.7781
 65/100 [==================>...........] - ETA: 10s - loss: 0.4862 - accuracy: 0.7777
 66/100 [==================>...........] - ETA: 10s - loss: 0.4875 - accuracy: 0.7780
 67/100 [===================>..........] - ETA: 10s - loss: 0.4888 - accuracy: 0.7769
 68/100 [===================>..........] - ETA: 9s - loss: 0.4884 - accuracy: 0.7765 
 69/100 [===================>..........] - ETA: 9s - loss: 0.4873 - accuracy: 0.7783
 70/100 [====================>.........] - ETA: 9s - loss: 0.4894 - accuracy: 0.7764
 71/100 [====================>.........] - ETA: 8s - loss: 0.4882 - accuracy: 0.7775
 72/100 [====================>.........] - ETA: 8s - loss: 0.4892 - accuracy: 0.7771
 73/100 [====================>.........] - ETA: 8s - loss: 0.4888 - accuracy: 0.7781
 74/100 [=====================>........] - ETA: 7s - loss: 0.4871 - accuracy: 0.7791
 75/100 [=====================>........] - ETA: 7s - loss: 0.4861 - accuracy: 0.7793
 76/100 [=====================>........] - ETA: 7s - loss: 0.4856 - accuracy: 0.7789
 77/100 [======================>.......] - ETA: 7s - loss: 0.4854 - accuracy: 0.7799
 78/100 [======================>.......] - ETA: 6s - loss: 0.4851 - accuracy: 0.7814
 79/100 [======================>.......] - ETA: 6s - loss: 0.4832 - accuracy: 0.7829
 80/100 [=======================>......] - ETA: 6s - loss: 0.4823 - accuracy: 0.7844
 81/100 [=======================>......] - ETA: 5s - loss: 0.4818 - accuracy: 0.7852
 82/100 [=======================>......] - ETA: 5s - loss: 0.4816 - accuracy: 0.7848
 83/100 [=======================>......] - ETA: 5s - loss: 0.4801 - accuracy: 0.7855
 84/100 [========================>.....] - ETA: 4s - loss: 0.4822 - accuracy: 0.7857
 85/100 [========================>.....] - ETA: 4s - loss: 0.4808 - accuracy: 0.7853
 86/100 [========================>.....] - ETA: 4s - loss: 0.4829 - accuracy: 0.7837
 87/100 [=========================>....] - ETA: 4s - loss: 0.4841 - accuracy: 0.7822
 88/100 [=========================>....] - ETA: 3s - loss: 0.4841 - accuracy: 0.7818
 89/100 [=========================>....] - ETA: 3s - loss: 0.4835 - accuracy: 0.7826
 90/100 [==========================>...] - ETA: 3s - loss: 0.4856 - accuracy: 0.7811
 91/100 [==========================>...] - ETA: 2s - loss: 0.4846 - accuracy: 0.7813
 92/100 [==========================>...] - ETA: 2s - loss: 0.4853 - accuracy: 0.7810
 93/100 [==========================>...] - ETA: 2s - loss: 0.4860 - accuracy: 0.7817
 94/100 [===========================>..] - ETA: 1s - loss: 0.4878 - accuracy: 0.7793
 95/100 [===========================>..] - ETA: 1s - loss: 0.4868 - accuracy: 0.7795
 96/100 [===========================>..] - ETA: 1s - loss: 0.4867 - accuracy: 0.7797
 97/100 [============================>.] - ETA: 0s - loss: 0.4867 - accuracy: 0.7799
 98/100 [============================>.] - ETA: 0s - loss: 0.4858 - accuracy: 0.7806
 99/100 [============================>.] - ETA: 0s - loss: 0.4863 - accuracy: 0.7808
100/100 [==============================] - 32s 317ms/step - loss: 0.4869 - accuracy: 0.7795

100/100 [==============================] - 36s 359ms/step - loss: 0.4869 - accuracy: 0.7795 - val_loss: 0.5566 - val_accuracy: 0.7050
Epoch 8/30

  1/100 [..............................] - ETA: 27s - loss: 0.7069 - accuracy: 0.5000
  2/100 [..............................] - ETA: 22s - loss: 0.6168 - accuracy: 0.5750
  3/100 [..............................] - ETA: 21s - loss: 0.5329 - accuracy: 0.7000
  4/100 [>.............................] - ETA: 21s - loss: 0.4913 - accuracy: 0.7500
  5/100 [>.............................] - ETA: 22s - loss: 0.5120 - accuracy: 0.7200
  6/100 [>.............................] - ETA: 22s - loss: 0.4958 - accuracy: 0.7333
  7/100 [=>............................] - ETA: 21s - loss: 0.5050 - accuracy: 0.7357
  8/100 [=>............................] - ETA: 21s - loss: 0.5485 - accuracy: 0.7000
  9/100 [=>............................] - ETA: 21s - loss: 0.5378 - accuracy: 0.7111
 10/100 [==>...........................] - ETA: 20s - loss: 0.5275 - accuracy: 0.7250
 11/100 [==>...........................] - ETA: 20s - loss: 0.5214 - accuracy: 0.7227
 12/100 [==>...........................] - ETA: 20s - loss: 0.5209 - accuracy: 0.7167
 13/100 [==>...........................] - ETA: 19s - loss: 0.5124 - accuracy: 0.7269
 14/100 [===>..........................] - ETA: 19s - loss: 0.5205 - accuracy: 0.7179
 15/100 [===>..........................] - ETA: 19s - loss: 0.5153 - accuracy: 0.7267
 16/100 [===>..........................] - ETA: 19s - loss: 0.5134 - accuracy: 0.7344
 17/100 [====>.........................] - ETA: 18s - loss: 0.5088 - accuracy: 0.7353
 18/100 [====>.........................] - ETA: 18s - loss: 0.5088 - accuracy: 0.7333
 19/100 [====>.........................] - ETA: 18s - loss: 0.5148 - accuracy: 0.7263
 20/100 [=====>........................] - ETA: 18s - loss: 0.5134 - accuracy: 0.7325
 21/100 [=====>........................] - ETA: 17s - loss: 0.5089 - accuracy: 0.7333
 22/100 [=====>........................] - ETA: 17s - loss: 0.5092 - accuracy: 0.7318
 23/100 [=====>........................] - ETA: 17s - loss: 0.5044 - accuracy: 0.7370
 24/100 [======>.......................] - ETA: 17s - loss: 0.5039 - accuracy: 0.7396
 25/100 [======>.......................] - ETA: 16s - loss: 0.4978 - accuracy: 0.7440
 26/100 [======>.......................] - ETA: 17s - loss: 0.4948 - accuracy: 0.7442
 27/100 [=======>......................] - ETA: 17s - loss: 0.4971 - accuracy: 0.7370
 28/100 [=======>......................] - ETA: 17s - loss: 0.4999 - accuracy: 0.7357
 29/100 [=======>......................] - ETA: 17s - loss: 0.4984 - accuracy: 0.7379
 30/100 [========>.....................] - ETA: 17s - loss: 0.4971 - accuracy: 0.7367
 31/100 [========>.....................] - ETA: 17s - loss: 0.4988 - accuracy: 0.7355
 32/100 [========>.....................] - ETA: 17s - loss: 0.4960 - accuracy: 0.7406
 33/100 [========>.....................] - ETA: 17s - loss: 0.4948 - accuracy: 0.7424
 34/100 [=========>....................] - ETA: 17s - loss: 0.4908 - accuracy: 0.7471
 35/100 [=========>....................] - ETA: 17s - loss: 0.4948 - accuracy: 0.7443
 36/100 [=========>....................] - ETA: 16s - loss: 0.4990 - accuracy: 0.7417
 37/100 [==========>...................] - ETA: 16s - loss: 0.4998 - accuracy: 0.7432
 38/100 [==========>...................] - ETA: 16s - loss: 0.5000 - accuracy: 0.7447
 39/100 [==========>...................] - ETA: 16s - loss: 0.4982 - accuracy: 0.7449
 40/100 [===========>..................] - ETA: 16s - loss: 0.4918 - accuracy: 0.7500
 41/100 [===========>..................] - ETA: 16s - loss: 0.4879 - accuracy: 0.7549
 42/100 [===========>..................] - ETA: 16s - loss: 0.4860 - accuracy: 0.7571
 43/100 [===========>..................] - ETA: 15s - loss: 0.4848 - accuracy: 0.7581
 44/100 [============>.................] - ETA: 15s - loss: 0.4860 - accuracy: 0.7568
 45/100 [============>.................] - ETA: 15s - loss: 0.4835 - accuracy: 0.7600
 46/100 [============>.................] - ETA: 15s - loss: 0.4867 - accuracy: 0.7587
 47/100 [=============>................] - ETA: 15s - loss: 0.4873 - accuracy: 0.7596
 48/100 [=============>................] - ETA: 14s - loss: 0.4878 - accuracy: 0.7583
 49/100 [=============>................] - ETA: 14s - loss: 0.4841 - accuracy: 0.7622
 50/100 [==============>...............] - ETA: 14s - loss: 0.4837 - accuracy: 0.7630
 51/100 [==============>...............] - ETA: 14s - loss: 0.4836 - accuracy: 0.7627
 52/100 [==============>...............] - ETA: 13s - loss: 0.4819 - accuracy: 0.7644
 53/100 [==============>...............] - ETA: 13s - loss: 0.4829 - accuracy: 0.7642
 54/100 [===============>..............] - ETA: 13s - loss: 0.4848 - accuracy: 0.7630
 55/100 [===============>..............] - ETA: 13s - loss: 0.4814 - accuracy: 0.7645
 56/100 [===============>..............] - ETA: 12s - loss: 0.4768 - accuracy: 0.7688
 57/100 [================>.............] - ETA: 12s - loss: 0.4765 - accuracy: 0.7684
 58/100 [================>.............] - ETA: 12s - loss: 0.4745 - accuracy: 0.7690
 59/100 [================>.............] - ETA: 12s - loss: 0.4726 - accuracy: 0.7720
 60/100 [=================>............] - ETA: 11s - loss: 0.4756 - accuracy: 0.7717
 61/100 [=================>............] - ETA: 11s - loss: 0.4733 - accuracy: 0.7738
 62/100 [=================>............] - ETA: 11s - loss: 0.4711 - accuracy: 0.7750
 63/100 [=================>............] - ETA: 11s - loss: 0.4679 - accuracy: 0.7778
 64/100 [==================>...........] - ETA: 10s - loss: 0.4683 - accuracy: 0.7781
 65/100 [==================>...........] - ETA: 10s - loss: 0.4693 - accuracy: 0.7785
 66/100 [==================>...........] - ETA: 10s - loss: 0.4691 - accuracy: 0.7795
 67/100 [===================>..........] - ETA: 9s - loss: 0.4716 - accuracy: 0.7806 
 68/100 [===================>..........] - ETA: 9s - loss: 0.4713 - accuracy: 0.7794
 69/100 [===================>..........] - ETA: 9s - loss: 0.4688 - accuracy: 0.7812
 70/100 [====================>.........] - ETA: 9s - loss: 0.4676 - accuracy: 0.7829
 71/100 [====================>.........] - ETA: 8s - loss: 0.4687 - accuracy: 0.7831
 72/100 [====================>.........] - ETA: 8s - loss: 0.4670 - accuracy: 0.7847
 73/100 [====================>.........] - ETA: 8s - loss: 0.4670 - accuracy: 0.7842
 74/100 [=====================>........] - ETA: 7s - loss: 0.4696 - accuracy: 0.7818
 75/100 [=====================>........] - ETA: 7s - loss: 0.4703 - accuracy: 0.7820
 76/100 [=====================>........] - ETA: 7s - loss: 0.4699 - accuracy: 0.7816
 77/100 [======================>.......] - ETA: 7s - loss: 0.4697 - accuracy: 0.7805
 78/100 [======================>.......] - ETA: 6s - loss: 0.4683 - accuracy: 0.7814
 79/100 [======================>.......] - ETA: 6s - loss: 0.4694 - accuracy: 0.7797
 80/100 [=======================>......] - ETA: 6s - loss: 0.4691 - accuracy: 0.7800
 81/100 [=======================>......] - ETA: 5s - loss: 0.4695 - accuracy: 0.7796
 82/100 [=======================>......] - ETA: 5s - loss: 0.4713 - accuracy: 0.7780
 83/100 [=======================>......] - ETA: 5s - loss: 0.4707 - accuracy: 0.7795
 84/100 [========================>.....] - ETA: 4s - loss: 0.4699 - accuracy: 0.7798
 85/100 [========================>.....] - ETA: 4s - loss: 0.4693 - accuracy: 0.7812
 86/100 [========================>.....] - ETA: 4s - loss: 0.4682 - accuracy: 0.7820
 87/100 [=========================>....] - ETA: 4s - loss: 0.4696 - accuracy: 0.7810
 88/100 [=========================>....] - ETA: 3s - loss: 0.4695 - accuracy: 0.7807
 89/100 [=========================>....] - ETA: 3s - loss: 0.4697 - accuracy: 0.7803
 90/100 [==========================>...] - ETA: 3s - loss: 0.4691 - accuracy: 0.7800
 91/100 [==========================>...] - ETA: 2s - loss: 0.4680 - accuracy: 0.7813
 92/100 [==========================>...] - ETA: 2s - loss: 0.4680 - accuracy: 0.7810
 93/100 [==========================>...] - ETA: 2s - loss: 0.4669 - accuracy: 0.7812
 94/100 [===========================>..] - ETA: 1s - loss: 0.4659 - accuracy: 0.7814
 95/100 [===========================>..] - ETA: 1s - loss: 0.4664 - accuracy: 0.7805
 96/100 [===========================>..] - ETA: 1s - loss: 0.4657 - accuracy: 0.7807
 97/100 [============================>.] - ETA: 0s - loss: 0.4685 - accuracy: 0.7794
 98/100 [============================>.] - ETA: 0s - loss: 0.4682 - accuracy: 0.7796
 99/100 [============================>.] - ETA: 0s - loss: 0.4673 - accuracy: 0.7793
100/100 [==============================] - 32s 316ms/step - loss: 0.4679 - accuracy: 0.7790

100/100 [==============================] - 36s 358ms/step - loss: 0.4679 - accuracy: 0.7790 - val_loss: 0.5756 - val_accuracy: 0.7050
Epoch 9/30

  1/100 [..............................] - ETA: 29s - loss: 0.5327 - accuracy: 0.7500
  2/100 [..............................] - ETA: 20s - loss: 0.4542 - accuracy: 0.8000
  3/100 [..............................] - ETA: 21s - loss: 0.4641 - accuracy: 0.8000
  4/100 [>.............................] - ETA: 21s - loss: 0.4381 - accuracy: 0.8250
  5/100 [>.............................] - ETA: 21s - loss: 0.4330 - accuracy: 0.8200
  6/100 [>.............................] - ETA: 20s - loss: 0.4236 - accuracy: 0.8167
  7/100 [=>............................] - ETA: 20s - loss: 0.4180 - accuracy: 0.8143
  8/100 [=>............................] - ETA: 20s - loss: 0.3932 - accuracy: 0.8375
  9/100 [=>............................] - ETA: 20s - loss: 0.4041 - accuracy: 0.8444
 10/100 [==>...........................] - ETA: 20s - loss: 0.4008 - accuracy: 0.8500
 11/100 [==>...........................] - ETA: 19s - loss: 0.3904 - accuracy: 0.8545
 12/100 [==>...........................] - ETA: 20s - loss: 0.4255 - accuracy: 0.8333
 13/100 [==>...........................] - ETA: 19s - loss: 0.4240 - accuracy: 0.8385
 14/100 [===>..........................] - ETA: 19s - loss: 0.4180 - accuracy: 0.8429
 15/100 [===>..........................] - ETA: 19s - loss: 0.4344 - accuracy: 0.8333
 16/100 [===>..........................] - ETA: 19s - loss: 0.4407 - accuracy: 0.8219
 17/100 [====>.........................] - ETA: 18s - loss: 0.4502 - accuracy: 0.8059
 18/100 [====>.........................] - ETA: 18s - loss: 0.4626 - accuracy: 0.8000
 19/100 [====>.........................] - ETA: 18s - loss: 0.4591 - accuracy: 0.8026
 20/100 [=====>........................] - ETA: 17s - loss: 0.4613 - accuracy: 0.8000
 21/100 [=====>........................] - ETA: 17s - loss: 0.4582 - accuracy: 0.8048
 22/100 [=====>........................] - ETA: 17s - loss: 0.4519 - accuracy: 0.8091
 23/100 [=====>........................] - ETA: 17s - loss: 0.4472 - accuracy: 0.8109
 24/100 [======>.......................] - ETA: 17s - loss: 0.4468 - accuracy: 0.8125
 25/100 [======>.......................] - ETA: 17s - loss: 0.4425 - accuracy: 0.8100
 26/100 [======>.......................] - ETA: 17s - loss: 0.4416 - accuracy: 0.8096
 27/100 [=======>......................] - ETA: 17s - loss: 0.4430 - accuracy: 0.8111
 28/100 [=======>......................] - ETA: 17s - loss: 0.4420 - accuracy: 0.8107
 29/100 [=======>......................] - ETA: 17s - loss: 0.4466 - accuracy: 0.8069
 30/100 [========>.....................] - ETA: 17s - loss: 0.4496 - accuracy: 0.8033
 31/100 [========>.....................] - ETA: 17s - loss: 0.4497 - accuracy: 0.8032
 32/100 [========>.....................] - ETA: 17s - loss: 0.4469 - accuracy: 0.8078
 33/100 [========>.....................] - ETA: 17s - loss: 0.4454 - accuracy: 0.8061
 34/100 [=========>....................] - ETA: 17s - loss: 0.4404 - accuracy: 0.8088
 35/100 [=========>....................] - ETA: 17s - loss: 0.4408 - accuracy: 0.8057
 36/100 [=========>....................] - ETA: 17s - loss: 0.4403 - accuracy: 0.8056
 37/100 [==========>...................] - ETA: 17s - loss: 0.4399 - accuracy: 0.8041
 38/100 [==========>...................] - ETA: 17s - loss: 0.4391 - accuracy: 0.8053
 39/100 [==========>...................] - ETA: 17s - loss: 0.4368 - accuracy: 0.8051
 40/100 [===========>..................] - ETA: 16s - loss: 0.4420 - accuracy: 0.8025
 41/100 [===========>..................] - ETA: 16s - loss: 0.4454 - accuracy: 0.8012
 42/100 [===========>..................] - ETA: 16s - loss: 0.4419 - accuracy: 0.8048
 43/100 [===========>..................] - ETA: 16s - loss: 0.4383 - accuracy: 0.8070
 44/100 [============>.................] - ETA: 16s - loss: 0.4365 - accuracy: 0.8080
 45/100 [============>.................] - ETA: 16s - loss: 0.4360 - accuracy: 0.8067
 46/100 [============>.................] - ETA: 15s - loss: 0.4351 - accuracy: 0.8076
 47/100 [=============>................] - ETA: 15s - loss: 0.4385 - accuracy: 0.8064
 48/100 [=============>................] - ETA: 15s - loss: 0.4379 - accuracy: 0.8052
 49/100 [=============>................] - ETA: 15s - loss: 0.4366 - accuracy: 0.8051
 50/100 [==============>...............] - ETA: 14s - loss: 0.4378 - accuracy: 0.8040
 51/100 [==============>...............] - ETA: 14s - loss: 0.4370 - accuracy: 0.8049
 52/100 [==============>...............] - ETA: 14s - loss: 0.4397 - accuracy: 0.8019
 53/100 [==============>...............] - ETA: 14s - loss: 0.4405 - accuracy: 0.8000
 54/100 [===============>..............] - ETA: 13s - loss: 0.4402 - accuracy: 0.8000
 55/100 [===============>..............] - ETA: 13s - loss: 0.4411 - accuracy: 0.8000
 56/100 [===============>..............] - ETA: 13s - loss: 0.4429 - accuracy: 0.7991
 57/100 [================>.............] - ETA: 13s - loss: 0.4454 - accuracy: 0.7965
 58/100 [================>.............] - ETA: 12s - loss: 0.4447 - accuracy: 0.7966
 59/100 [================>.............] - ETA: 12s - loss: 0.4460 - accuracy: 0.7941
 60/100 [=================>............] - ETA: 12s - loss: 0.4498 - accuracy: 0.7917
 61/100 [=================>............] - ETA: 11s - loss: 0.4509 - accuracy: 0.7910
 62/100 [=================>............] - ETA: 11s - loss: 0.4490 - accuracy: 0.7919
 63/100 [=================>............] - ETA: 11s - loss: 0.4477 - accuracy: 0.7921
 64/100 [==================>...........] - ETA: 11s - loss: 0.4507 - accuracy: 0.7906
 65/100 [==================>...........] - ETA: 10s - loss: 0.4504 - accuracy: 0.7900
 66/100 [==================>...........] - ETA: 10s - loss: 0.4491 - accuracy: 0.7909
 67/100 [===================>..........] - ETA: 10s - loss: 0.4501 - accuracy: 0.7888
 68/100 [===================>..........] - ETA: 9s - loss: 0.4517 - accuracy: 0.7868 
 69/100 [===================>..........] - ETA: 9s - loss: 0.4504 - accuracy: 0.7877
 70/100 [====================>.........] - ETA: 9s - loss: 0.4499 - accuracy: 0.7886
 71/100 [====================>.........] - ETA: 9s - loss: 0.4490 - accuracy: 0.7887
 72/100 [====================>.........] - ETA: 8s - loss: 0.4483 - accuracy: 0.7882
 73/100 [====================>.........] - ETA: 8s - loss: 0.4485 - accuracy: 0.7884
 74/100 [=====================>........] - ETA: 8s - loss: 0.4495 - accuracy: 0.7878
 75/100 [=====================>........] - ETA: 7s - loss: 0.4477 - accuracy: 0.7887
 76/100 [=====================>........] - ETA: 7s - loss: 0.4467 - accuracy: 0.7895
 77/100 [======================>.......] - ETA: 7s - loss: 0.4452 - accuracy: 0.7903
 78/100 [======================>.......] - ETA: 6s - loss: 0.4452 - accuracy: 0.7897
 79/100 [======================>.......] - ETA: 6s - loss: 0.4496 - accuracy: 0.7873
 80/100 [=======================>......] - ETA: 6s - loss: 0.4496 - accuracy: 0.7881
 81/100 [=======================>......] - ETA: 6s - loss: 0.4482 - accuracy: 0.7895
 82/100 [=======================>......] - ETA: 5s - loss: 0.4474 - accuracy: 0.7896
 83/100 [=======================>......] - ETA: 5s - loss: 0.4462 - accuracy: 0.7904
 84/100 [========================>.....] - ETA: 5s - loss: 0.4460 - accuracy: 0.7893
 85/100 [========================>.....] - ETA: 4s - loss: 0.4445 - accuracy: 0.7900
 86/100 [========================>.....] - ETA: 4s - loss: 0.4463 - accuracy: 0.7878
 87/100 [=========================>....] - ETA: 4s - loss: 0.4468 - accuracy: 0.7874
 88/100 [=========================>....] - ETA: 3s - loss: 0.4474 - accuracy: 0.7869
 89/100 [=========================>....] - ETA: 3s - loss: 0.4476 - accuracy: 0.7865
 90/100 [==========================>...] - ETA: 3s - loss: 0.4477 - accuracy: 0.7861
 91/100 [==========================>...] - ETA: 2s - loss: 0.4474 - accuracy: 0.7863
 92/100 [==========================>...] - ETA: 2s - loss: 0.4449 - accuracy: 0.7886
 93/100 [==========================>...] - ETA: 2s - loss: 0.4438 - accuracy: 0.7903
 94/100 [===========================>..] - ETA: 1s - loss: 0.4435 - accuracy: 0.7910
 95/100 [===========================>..] - ETA: 1s - loss: 0.4427 - accuracy: 0.7911
 96/100 [===========================>..] - ETA: 1s - loss: 0.4425 - accuracy: 0.7917
 97/100 [============================>.] - ETA: 0s - loss: 0.4406 - accuracy: 0.7933
 98/100 [============================>.] - ETA: 0s - loss: 0.4408 - accuracy: 0.7934
 99/100 [============================>.] - ETA: 0s - loss: 0.4406 - accuracy: 0.7934
100/100 [==============================] - 33s 326ms/step - loss: 0.4392 - accuracy: 0.7940

100/100 [==============================] - 36s 366ms/step - loss: 0.4392 - accuracy: 0.7940 - val_loss: 0.5820 - val_accuracy: 0.7080
Epoch 10/30

  1/100 [..............................] - ETA: 25s - loss: 0.4587 - accuracy: 0.6500
  2/100 [..............................] - ETA: 20s - loss: 0.4176 - accuracy: 0.7250
  3/100 [..............................] - ETA: 20s - loss: 0.5260 - accuracy: 0.6667
  4/100 [>.............................] - ETA: 20s - loss: 0.5793 - accuracy: 0.6375
  5/100 [>.............................] - ETA: 20s - loss: 0.5568 - accuracy: 0.6900
  6/100 [>.............................] - ETA: 20s - loss: 0.5141 - accuracy: 0.7250
  7/100 [=>............................] - ETA: 20s - loss: 0.5109 - accuracy: 0.7214
  8/100 [=>............................] - ETA: 19s - loss: 0.5070 - accuracy: 0.7312
  9/100 [=>............................] - ETA: 19s - loss: 0.4870 - accuracy: 0.7444
 10/100 [==>...........................] - ETA: 19s - loss: 0.4689 - accuracy: 0.7550
 11/100 [==>...........................] - ETA: 19s - loss: 0.4492 - accuracy: 0.7727
 12/100 [==>...........................] - ETA: 19s - loss: 0.4373 - accuracy: 0.7792
 13/100 [==>...........................] - ETA: 18s - loss: 0.4401 - accuracy: 0.7808
 14/100 [===>..........................] - ETA: 18s - loss: 0.4336 - accuracy: 0.7857
 15/100 [===>..........................] - ETA: 18s - loss: 0.4315 - accuracy: 0.7867
 16/100 [===>..........................] - ETA: 18s - loss: 0.4251 - accuracy: 0.7937
 17/100 [====>.........................] - ETA: 17s - loss: 0.4268 - accuracy: 0.7912
 18/100 [====>.........................] - ETA: 17s - loss: 0.4249 - accuracy: 0.7944
 19/100 [====>.........................] - ETA: 17s - loss: 0.4235 - accuracy: 0.7947
 20/100 [=====>........................] - ETA: 17s - loss: 0.4195 - accuracy: 0.7975
 21/100 [=====>........................] - ETA: 17s - loss: 0.4226 - accuracy: 0.7976
 22/100 [=====>........................] - ETA: 16s - loss: 0.4177 - accuracy: 0.8000
 23/100 [=====>........................] - ETA: 16s - loss: 0.4152 - accuracy: 0.8022
 24/100 [======>.......................] - ETA: 16s - loss: 0.4170 - accuracy: 0.8000
 25/100 [======>.......................] - ETA: 16s - loss: 0.4259 - accuracy: 0.7920
 26/100 [======>.......................] - ETA: 16s - loss: 0.4245 - accuracy: 0.7942
 27/100 [=======>......................] - ETA: 16s - loss: 0.4206 - accuracy: 0.8000
 28/100 [=======>......................] - ETA: 16s - loss: 0.4220 - accuracy: 0.7982
 29/100 [=======>......................] - ETA: 16s - loss: 0.4200 - accuracy: 0.8000
 30/100 [========>.....................] - ETA: 16s - loss: 0.4187 - accuracy: 0.8033
 31/100 [========>.....................] - ETA: 16s - loss: 0.4157 - accuracy: 0.8065
 32/100 [========>.....................] - ETA: 16s - loss: 0.4152 - accuracy: 0.8078
 33/100 [========>.....................] - ETA: 16s - loss: 0.4150 - accuracy: 0.8091
 34/100 [=========>....................] - ETA: 16s - loss: 0.4081 - accuracy: 0.8147
 35/100 [=========>....................] - ETA: 16s - loss: 0.4062 - accuracy: 0.8171
 36/100 [=========>....................] - ETA: 16s - loss: 0.4044 - accuracy: 0.8167
 37/100 [==========>...................] - ETA: 16s - loss: 0.4086 - accuracy: 0.8135
 38/100 [==========>...................] - ETA: 16s - loss: 0.4092 - accuracy: 0.8132
 39/100 [==========>...................] - ETA: 16s - loss: 0.4072 - accuracy: 0.8128
 40/100 [===========>..................] - ETA: 15s - loss: 0.4085 - accuracy: 0.8125
 41/100 [===========>..................] - ETA: 15s - loss: 0.4104 - accuracy: 0.8134
 42/100 [===========>..................] - ETA: 15s - loss: 0.4119 - accuracy: 0.8107
 43/100 [===========>..................] - ETA: 15s - loss: 0.4089 - accuracy: 0.8151
 44/100 [============>.................] - ETA: 15s - loss: 0.4077 - accuracy: 0.8159
 45/100 [============>.................] - ETA: 15s - loss: 0.4048 - accuracy: 0.8189
 46/100 [============>.................] - ETA: 14s - loss: 0.4108 - accuracy: 0.8163
 47/100 [=============>................] - ETA: 14s - loss: 0.4119 - accuracy: 0.8138
 48/100 [=============>................] - ETA: 14s - loss: 0.4111 - accuracy: 0.8135
 49/100 [=============>................] - ETA: 14s - loss: 0.4130 - accuracy: 0.8112
 50/100 [==============>...............] - ETA: 14s - loss: 0.4131 - accuracy: 0.8120
 51/100 [==============>...............] - ETA: 14s - loss: 0.4150 - accuracy: 0.8108
 52/100 [==============>...............] - ETA: 13s - loss: 0.4141 - accuracy: 0.8096
 53/100 [==============>...............] - ETA: 13s - loss: 0.4158 - accuracy: 0.8085
 54/100 [===============>..............] - ETA: 13s - loss: 0.4166 - accuracy: 0.8083
 55/100 [===============>..............] - ETA: 13s - loss: 0.4158 - accuracy: 0.8091
 56/100 [===============>..............] - ETA: 12s - loss: 0.4169 - accuracy: 0.8071
 57/100 [================>.............] - ETA: 12s - loss: 0.4190 - accuracy: 0.8053
 58/100 [================>.............] - ETA: 12s - loss: 0.4189 - accuracy: 0.8060
 59/100 [================>.............] - ETA: 12s - loss: 0.4198 - accuracy: 0.8034
 60/100 [=================>............] - ETA: 11s - loss: 0.4190 - accuracy: 0.8050
 61/100 [=================>............] - ETA: 11s - loss: 0.4208 - accuracy: 0.8033
 62/100 [=================>............] - ETA: 11s - loss: 0.4202 - accuracy: 0.8048
 63/100 [=================>............] - ETA: 10s - loss: 0.4189 - accuracy: 0.8056
 64/100 [==================>...........] - ETA: 10s - loss: 0.4155 - accuracy: 0.8086
 65/100 [==================>...........] - ETA: 10s - loss: 0.4147 - accuracy: 0.8092
 66/100 [==================>...........] - ETA: 10s - loss: 0.4134 - accuracy: 0.8098
 67/100 [===================>..........] - ETA: 9s - loss: 0.4130 - accuracy: 0.8097 
 68/100 [===================>..........] - ETA: 9s - loss: 0.4132 - accuracy: 0.8096
 69/100 [===================>..........] - ETA: 9s - loss: 0.4121 - accuracy: 0.8094
 70/100 [====================>.........] - ETA: 9s - loss: 0.4122 - accuracy: 0.8093
 71/100 [====================>.........] - ETA: 8s - loss: 0.4103 - accuracy: 0.8113
 72/100 [====================>.........] - ETA: 8s - loss: 0.4111 - accuracy: 0.8111
 73/100 [====================>.........] - ETA: 8s - loss: 0.4120 - accuracy: 0.8110
 74/100 [=====================>........] - ETA: 7s - loss: 0.4130 - accuracy: 0.8088
 75/100 [=====================>........] - ETA: 7s - loss: 0.4135 - accuracy: 0.8100
 76/100 [=====================>........] - ETA: 7s - loss: 0.4133 - accuracy: 0.8105
 77/100 [======================>.......] - ETA: 7s - loss: 0.4134 - accuracy: 0.8097
 78/100 [======================>.......] - ETA: 6s - loss: 0.4132 - accuracy: 0.8096
 79/100 [======================>.......] - ETA: 6s - loss: 0.4132 - accuracy: 0.8101
 80/100 [=======================>......] - ETA: 6s - loss: 0.4141 - accuracy: 0.8094
 81/100 [=======================>......] - ETA: 5s - loss: 0.4158 - accuracy: 0.8099
 82/100 [=======================>......] - ETA: 5s - loss: 0.4159 - accuracy: 0.8098
 83/100 [=======================>......] - ETA: 5s - loss: 0.4155 - accuracy: 0.8102
 84/100 [========================>.....] - ETA: 4s - loss: 0.4160 - accuracy: 0.8101
 85/100 [========================>.....] - ETA: 4s - loss: 0.4153 - accuracy: 0.8112
 86/100 [========================>.....] - ETA: 4s - loss: 0.4151 - accuracy: 0.8105
 87/100 [=========================>....] - ETA: 4s - loss: 0.4154 - accuracy: 0.8103
 88/100 [=========================>....] - ETA: 3s - loss: 0.4151 - accuracy: 0.8108
 89/100 [=========================>....] - ETA: 3s - loss: 0.4158 - accuracy: 0.8107
 90/100 [==========================>...] - ETA: 3s - loss: 0.4146 - accuracy: 0.8117
 91/100 [==========================>...] - ETA: 2s - loss: 0.4171 - accuracy: 0.8104
 92/100 [==========================>...] - ETA: 2s - loss: 0.4160 - accuracy: 0.8114
 93/100 [==========================>...] - ETA: 2s - loss: 0.4162 - accuracy: 0.8118
 94/100 [===========================>..] - ETA: 1s - loss: 0.4161 - accuracy: 0.8112
 95/100 [===========================>..] - ETA: 1s - loss: 0.4169 - accuracy: 0.8105
 96/100 [===========================>..] - ETA: 1s - loss: 0.4170 - accuracy: 0.8109
 97/100 [============================>.] - ETA: 0s - loss: 0.4170 - accuracy: 0.8098
 98/100 [============================>.] - ETA: 0s - loss: 0.4178 - accuracy: 0.8087
 99/100 [============================>.] - ETA: 0s - loss: 0.4200 - accuracy: 0.8071
100/100 [==============================] - 32s 319ms/step - loss: 0.4204 - accuracy: 0.8065

100/100 [==============================] - 36s 363ms/step - loss: 0.4204 - accuracy: 0.8065 - val_loss: 0.6259 - val_accuracy: 0.6850
Epoch 11/30

  1/100 [..............................] - ETA: 29s - loss: 0.6634 - accuracy: 0.6500
  2/100 [..............................] - ETA: 24s - loss: 0.6005 - accuracy: 0.6750
  3/100 [..............................] - ETA: 23s - loss: 0.5458 - accuracy: 0.6833
  4/100 [>.............................] - ETA: 22s - loss: 0.4764 - accuracy: 0.7625
  5/100 [>.............................] - ETA: 21s - loss: 0.4551 - accuracy: 0.7900
  6/100 [>.............................] - ETA: 21s - loss: 0.4422 - accuracy: 0.8000
  7/100 [=>............................] - ETA: 21s - loss: 0.4195 - accuracy: 0.8143
  8/100 [=>............................] - ETA: 20s - loss: 0.4142 - accuracy: 0.8062
  9/100 [=>............................] - ETA: 20s - loss: 0.4188 - accuracy: 0.7944
 10/100 [==>...........................] - ETA: 20s - loss: 0.4265 - accuracy: 0.7850
 11/100 [==>...........................] - ETA: 20s - loss: 0.4169 - accuracy: 0.8000
 12/100 [==>...........................] - ETA: 20s - loss: 0.4063 - accuracy: 0.8167
 13/100 [==>...........................] - ETA: 19s - loss: 0.4130 - accuracy: 0.8115
 14/100 [===>..........................] - ETA: 19s - loss: 0.3971 - accuracy: 0.8250
 15/100 [===>..........................] - ETA: 19s - loss: 0.3925 - accuracy: 0.8300
 16/100 [===>..........................] - ETA: 19s - loss: 0.3919 - accuracy: 0.8219
 17/100 [====>.........................] - ETA: 18s - loss: 0.3924 - accuracy: 0.8235
 18/100 [====>.........................] - ETA: 18s - loss: 0.3962 - accuracy: 0.8222
 19/100 [====>.........................] - ETA: 18s - loss: 0.3853 - accuracy: 0.8316
 20/100 [=====>........................] - ETA: 18s - loss: 0.3914 - accuracy: 0.8250
 21/100 [=====>........................] - ETA: 17s - loss: 0.3882 - accuracy: 0.8310
 22/100 [=====>........................] - ETA: 17s - loss: 0.3893 - accuracy: 0.8273
 23/100 [=====>........................] - ETA: 17s - loss: 0.3839 - accuracy: 0.8348
 24/100 [======>.......................] - ETA: 17s - loss: 0.3861 - accuracy: 0.8354
 25/100 [======>.......................] - ETA: 17s - loss: 0.3834 - accuracy: 0.8380
 26/100 [======>.......................] - ETA: 18s - loss: 0.3810 - accuracy: 0.8385
 27/100 [=======>......................] - ETA: 18s - loss: 0.3834 - accuracy: 0.8389
 28/100 [=======>......................] - ETA: 18s - loss: 0.3924 - accuracy: 0.8339
 29/100 [=======>......................] - ETA: 18s - loss: 0.3872 - accuracy: 0.8397
 30/100 [========>.....................] - ETA: 18s - loss: 0.3932 - accuracy: 0.8317
 31/100 [========>.....................] - ETA: 18s - loss: 0.3969 - accuracy: 0.8274
 32/100 [========>.....................] - ETA: 18s - loss: 0.4043 - accuracy: 0.8219
 33/100 [========>.....................] - ETA: 18s - loss: 0.4101 - accuracy: 0.8182
 34/100 [=========>....................] - ETA: 18s - loss: 0.4094 - accuracy: 0.8176
 35/100 [=========>....................] - ETA: 18s - loss: 0.4090 - accuracy: 0.8171
 36/100 [=========>....................] - ETA: 17s - loss: 0.4128 - accuracy: 0.8139
 37/100 [==========>...................] - ETA: 17s - loss: 0.4132 - accuracy: 0.8135
 38/100 [==========>...................] - ETA: 17s - loss: 0.4099 - accuracy: 0.8158
 39/100 [==========>...................] - ETA: 17s - loss: 0.4048 - accuracy: 0.8192
 40/100 [===========>..................] - ETA: 17s - loss: 0.4065 - accuracy: 0.8200
 41/100 [===========>..................] - ETA: 17s - loss: 0.4097 - accuracy: 0.8171
 42/100 [===========>..................] - ETA: 16s - loss: 0.4061 - accuracy: 0.8214
 43/100 [===========>..................] - ETA: 16s - loss: 0.4102 - accuracy: 0.8198
 44/100 [============>.................] - ETA: 16s - loss: 0.4073 - accuracy: 0.8216
 45/100 [============>.................] - ETA: 16s - loss: 0.4104 - accuracy: 0.8189
 46/100 [============>.................] - ETA: 16s - loss: 0.4081 - accuracy: 0.8207
 47/100 [=============>................] - ETA: 15s - loss: 0.4044 - accuracy: 0.8234
 48/100 [=============>................] - ETA: 15s - loss: 0.4019 - accuracy: 0.8250
 49/100 [=============>................] - ETA: 15s - loss: 0.4004 - accuracy: 0.8265
 50/100 [==============>...............] - ETA: 15s - loss: 0.3995 - accuracy: 0.8270
 51/100 [==============>...............] - ETA: 14s - loss: 0.4003 - accuracy: 0.8255
 52/100 [==============>...............] - ETA: 14s - loss: 0.4077 - accuracy: 0.8202
 53/100 [==============>...............] - ETA: 14s - loss: 0.4050 - accuracy: 0.8226
 54/100 [===============>..............] - ETA: 14s - loss: 0.4052 - accuracy: 0.8222
 55/100 [===============>..............] - ETA: 13s - loss: 0.4020 - accuracy: 0.8245
 56/100 [===============>..............] - ETA: 13s - loss: 0.4081 - accuracy: 0.8205
 57/100 [================>.............] - ETA: 13s - loss: 0.4106 - accuracy: 0.8184
 58/100 [================>.............] - ETA: 12s - loss: 0.4101 - accuracy: 0.8190
 59/100 [================>.............] - ETA: 12s - loss: 0.4080 - accuracy: 0.8195
 60/100 [=================>............] - ETA: 12s - loss: 0.4059 - accuracy: 0.8200
 61/100 [=================>............] - ETA: 12s - loss: 0.4052 - accuracy: 0.8213
 62/100 [=================>............] - ETA: 11s - loss: 0.4046 - accuracy: 0.8218
 63/100 [=================>............] - ETA: 11s - loss: 0.4045 - accuracy: 0.8222
 64/100 [==================>...........] - ETA: 11s - loss: 0.4017 - accuracy: 0.8250
 65/100 [==================>...........] - ETA: 10s - loss: 0.4019 - accuracy: 0.8246
 66/100 [==================>...........] - ETA: 10s - loss: 0.4024 - accuracy: 0.8227
 67/100 [===================>..........] - ETA: 10s - loss: 0.3999 - accuracy: 0.8246
 68/100 [===================>..........] - ETA: 10s - loss: 0.3976 - accuracy: 0.8272
 69/100 [===================>..........] - ETA: 9s - loss: 0.3972 - accuracy: 0.8261 
 70/100 [====================>.........] - ETA: 9s - loss: 0.3975 - accuracy: 0.8243
 71/100 [====================>.........] - ETA: 9s - loss: 0.3982 - accuracy: 0.8225
 72/100 [====================>.........] - ETA: 8s - loss: 0.4016 - accuracy: 0.8208
 73/100 [====================>.........] - ETA: 8s - loss: 0.4009 - accuracy: 0.8219
 74/100 [=====================>........] - ETA: 8s - loss: 0.4023 - accuracy: 0.8209
 75/100 [=====================>........] - ETA: 7s - loss: 0.4009 - accuracy: 0.8227
 76/100 [=====================>........] - ETA: 7s - loss: 0.4002 - accuracy: 0.8230
 77/100 [======================>.......] - ETA: 7s - loss: 0.4005 - accuracy: 0.8234
 78/100 [======================>.......] - ETA: 7s - loss: 0.4001 - accuracy: 0.8237
 79/100 [======================>.......] - ETA: 6s - loss: 0.4034 - accuracy: 0.8215
 80/100 [=======================>......] - ETA: 6s - loss: 0.4065 - accuracy: 0.8200
 81/100 [=======================>......] - ETA: 6s - loss: 0.4056 - accuracy: 0.8204
 82/100 [=======================>......] - ETA: 5s - loss: 0.4074 - accuracy: 0.8189
 83/100 [=======================>......] - ETA: 5s - loss: 0.4061 - accuracy: 0.8193
 84/100 [========================>.....] - ETA: 5s - loss: 0.4057 - accuracy: 0.8202
 85/100 [========================>.....] - ETA: 4s - loss: 0.4060 - accuracy: 0.8200
 86/100 [========================>.....] - ETA: 4s - loss: 0.4056 - accuracy: 0.8209
 87/100 [=========================>....] - ETA: 4s - loss: 0.4061 - accuracy: 0.8207
 88/100 [=========================>....] - ETA: 3s - loss: 0.4055 - accuracy: 0.8210
 89/100 [=========================>....] - ETA: 3s - loss: 0.4042 - accuracy: 0.8225
 90/100 [==========================>...] - ETA: 3s - loss: 0.4039 - accuracy: 0.8222
 91/100 [==========================>...] - ETA: 2s - loss: 0.4034 - accuracy: 0.8231
 92/100 [==========================>...] - ETA: 2s - loss: 0.4031 - accuracy: 0.8228
 93/100 [==========================>...] - ETA: 2s - loss: 0.4043 - accuracy: 0.8220
 94/100 [===========================>..] - ETA: 1s - loss: 0.4033 - accuracy: 0.8223
 95/100 [===========================>..] - ETA: 1s - loss: 0.4033 - accuracy: 0.8226
 96/100 [===========================>..] - ETA: 1s - loss: 0.4022 - accuracy: 0.8240
 97/100 [============================>.] - ETA: 0s - loss: 0.4020 - accuracy: 0.8247
 98/100 [============================>.] - ETA: 0s - loss: 0.4011 - accuracy: 0.8250
 99/100 [============================>.] - ETA: 0s - loss: 0.3989 - accuracy: 0.8268
100/100 [==============================] - 33s 328ms/step - loss: 0.3982 - accuracy: 0.8275

100/100 [==============================] - 37s 371ms/step - loss: 0.3982 - accuracy: 0.8275 - val_loss: 0.6418 - val_accuracy: 0.6890
Epoch 12/30

  1/100 [..............................] - ETA: 26s - loss: 0.3688 - accuracy: 0.9000
  2/100 [..............................] - ETA: 22s - loss: 0.3070 - accuracy: 0.9500
  3/100 [..............................] - ETA: 22s - loss: 0.2857 - accuracy: 0.9500
  4/100 [>.............................] - ETA: 21s - loss: 0.3625 - accuracy: 0.8750
  5/100 [>.............................] - ETA: 21s - loss: 0.3411 - accuracy: 0.9000
  6/100 [>.............................] - ETA: 21s - loss: 0.3630 - accuracy: 0.8667
  7/100 [=>............................] - ETA: 21s - loss: 0.3714 - accuracy: 0.8500
  8/100 [=>............................] - ETA: 20s - loss: 0.3630 - accuracy: 0.8500
  9/100 [=>............................] - ETA: 20s - loss: 0.3643 - accuracy: 0.8500
 10/100 [==>...........................] - ETA: 20s - loss: 0.3604 - accuracy: 0.8500
 11/100 [==>...........................] - ETA: 20s - loss: 0.3552 - accuracy: 0.8500
 12/100 [==>...........................] - ETA: 19s - loss: 0.3506 - accuracy: 0.8583
 13/100 [==>...........................] - ETA: 19s - loss: 0.3549 - accuracy: 0.8462
 14/100 [===>..........................] - ETA: 19s - loss: 0.3677 - accuracy: 0.8393
 15/100 [===>..........................] - ETA: 19s - loss: 0.3676 - accuracy: 0.8400
 16/100 [===>..........................] - ETA: 18s - loss: 0.3591 - accuracy: 0.8469
 17/100 [====>.........................] - ETA: 18s - loss: 0.3599 - accuracy: 0.8441
 18/100 [====>.........................] - ETA: 18s - loss: 0.3566 - accuracy: 0.8444
 19/100 [====>.........................] - ETA: 17s - loss: 0.3575 - accuracy: 0.8447
 20/100 [=====>........................] - ETA: 17s - loss: 0.3589 - accuracy: 0.8475
 21/100 [=====>........................] - ETA: 17s - loss: 0.3585 - accuracy: 0.8500
 22/100 [=====>........................] - ETA: 17s - loss: 0.3527 - accuracy: 0.8523
 23/100 [=====>........................] - ETA: 16s - loss: 0.3576 - accuracy: 0.8478
 24/100 [======>.......................] - ETA: 16s - loss: 0.3639 - accuracy: 0.8438
 25/100 [======>.......................] - ETA: 16s - loss: 0.3636 - accuracy: 0.8420
 26/100 [======>.......................] - ETA: 16s - loss: 0.3669 - accuracy: 0.8385
 27/100 [=======>......................] - ETA: 17s - loss: 0.3660 - accuracy: 0.8389
 28/100 [=======>......................] - ETA: 17s - loss: 0.3687 - accuracy: 0.8375
 29/100 [=======>......................] - ETA: 17s - loss: 0.3660 - accuracy: 0.8397
 30/100 [========>.....................] - ETA: 17s - loss: 0.3654 - accuracy: 0.8417
 31/100 [========>.....................] - ETA: 17s - loss: 0.3635 - accuracy: 0.8419
 32/100 [========>.....................] - ETA: 17s - loss: 0.3590 - accuracy: 0.8438
 33/100 [========>.....................] - ETA: 17s - loss: 0.3571 - accuracy: 0.8470
 34/100 [=========>....................] - ETA: 17s - loss: 0.3590 - accuracy: 0.8441
 35/100 [=========>....................] - ETA: 17s - loss: 0.3695 - accuracy: 0.8386
 36/100 [=========>....................] - ETA: 17s - loss: 0.3679 - accuracy: 0.8417
 37/100 [==========>...................] - ETA: 17s - loss: 0.3715 - accuracy: 0.8378
 38/100 [==========>...................] - ETA: 17s - loss: 0.3701 - accuracy: 0.8395
 39/100 [==========>...................] - ETA: 16s - loss: 0.3695 - accuracy: 0.8397
 40/100 [===========>..................] - ETA: 16s - loss: 0.3664 - accuracy: 0.8413
 41/100 [===========>..................] - ETA: 16s - loss: 0.3653 - accuracy: 0.8427
 42/100 [===========>..................] - ETA: 16s - loss: 0.3643 - accuracy: 0.8417
 43/100 [===========>..................] - ETA: 16s - loss: 0.3626 - accuracy: 0.8442
 44/100 [============>.................] - ETA: 16s - loss: 0.3620 - accuracy: 0.8432
 45/100 [============>.................] - ETA: 16s - loss: 0.3596 - accuracy: 0.8444
 46/100 [============>.................] - ETA: 16s - loss: 0.3584 - accuracy: 0.8446
 47/100 [=============>................] - ETA: 16s - loss: 0.3576 - accuracy: 0.8468
 48/100 [=============>................] - ETA: 15s - loss: 0.3573 - accuracy: 0.8469
 49/100 [=============>................] - ETA: 15s - loss: 0.3551 - accuracy: 0.8480
 50/100 [==============>...............] - ETA: 15s - loss: 0.3520 - accuracy: 0.8500
 51/100 [==============>...............] - ETA: 15s - loss: 0.3528 - accuracy: 0.8490
 52/100 [==============>...............] - ETA: 15s - loss: 0.3545 - accuracy: 0.8481
 53/100 [==============>...............] - ETA: 15s - loss: 0.3548 - accuracy: 0.8472
 54/100 [===============>..............] - ETA: 14s - loss: 0.3536 - accuracy: 0.8481
 55/100 [===============>..............] - ETA: 14s - loss: 0.3529 - accuracy: 0.8491
 56/100 [===============>..............] - ETA: 14s - loss: 0.3513 - accuracy: 0.8500
 57/100 [================>.............] - ETA: 13s - loss: 0.3531 - accuracy: 0.8491
 58/100 [================>.............] - ETA: 13s - loss: 0.3514 - accuracy: 0.8491
 59/100 [================>.............] - ETA: 13s - loss: 0.3508 - accuracy: 0.8500
 60/100 [=================>............] - ETA: 13s - loss: 0.3518 - accuracy: 0.8492
 61/100 [=================>............] - ETA: 12s - loss: 0.3502 - accuracy: 0.8508
 62/100 [=================>............] - ETA: 12s - loss: 0.3483 - accuracy: 0.8524
 63/100 [=================>............] - ETA: 12s - loss: 0.3494 - accuracy: 0.8532
 64/100 [==================>...........] - ETA: 12s - loss: 0.3494 - accuracy: 0.8531
 65/100 [==================>...........] - ETA: 11s - loss: 0.3480 - accuracy: 0.8546
 66/100 [==================>...........] - ETA: 11s - loss: 0.3462 - accuracy: 0.8568
 67/100 [===================>..........] - ETA: 11s - loss: 0.3495 - accuracy: 0.8552
 68/100 [===================>..........] - ETA: 10s - loss: 0.3514 - accuracy: 0.8544
 69/100 [===================>..........] - ETA: 10s - loss: 0.3541 - accuracy: 0.8529
 70/100 [====================>.........] - ETA: 10s - loss: 0.3547 - accuracy: 0.8521
 71/100 [====================>.........] - ETA: 9s - loss: 0.3576 - accuracy: 0.8521 
 72/100 [====================>.........] - ETA: 9s - loss: 0.3583 - accuracy: 0.8507
 73/100 [====================>.........] - ETA: 9s - loss: 0.3592 - accuracy: 0.8500
 74/100 [=====================>........] - ETA: 9s - loss: 0.3619 - accuracy: 0.8493
 75/100 [=====================>........] - ETA: 8s - loss: 0.3643 - accuracy: 0.8480
 76/100 [=====================>........] - ETA: 8s - loss: 0.3668 - accuracy: 0.8461
 77/100 [======================>.......] - ETA: 8s - loss: 0.3654 - accuracy: 0.8474
 78/100 [======================>.......] - ETA: 7s - loss: 0.3634 - accuracy: 0.8487
 79/100 [======================>.......] - ETA: 7s - loss: 0.3630 - accuracy: 0.8487
 80/100 [=======================>......] - ETA: 7s - loss: 0.3630 - accuracy: 0.8481
 81/100 [=======================>......] - ETA: 6s - loss: 0.3624 - accuracy: 0.8488
 82/100 [=======================>......] - ETA: 6s - loss: 0.3625 - accuracy: 0.8488
 83/100 [=======================>......] - ETA: 6s - loss: 0.3645 - accuracy: 0.8476
 84/100 [========================>.....] - ETA: 5s - loss: 0.3647 - accuracy: 0.8464
 85/100 [========================>.....] - ETA: 5s - loss: 0.3640 - accuracy: 0.8471
 86/100 [========================>.....] - ETA: 5s - loss: 0.3625 - accuracy: 0.8477
 87/100 [=========================>....] - ETA: 4s - loss: 0.3644 - accuracy: 0.8466
 88/100 [=========================>....] - ETA: 4s - loss: 0.3674 - accuracy: 0.8449
 89/100 [=========================>....] - ETA: 3s - loss: 0.3663 - accuracy: 0.8461
 90/100 [==========================>...] - ETA: 3s - loss: 0.3677 - accuracy: 0.8450
 91/100 [==========================>...] - ETA: 3s - loss: 0.3659 - accuracy: 0.8467
 92/100 [==========================>...] - ETA: 2s - loss: 0.3650 - accuracy: 0.8484
 93/100 [==========================>...] - ETA: 2s - loss: 0.3664 - accuracy: 0.8478
 94/100 [===========================>..] - ETA: 2s - loss: 0.3662 - accuracy: 0.8479
 95/100 [===========================>..] - ETA: 1s - loss: 0.3655 - accuracy: 0.8484
 96/100 [===========================>..] - ETA: 1s - loss: 0.3656 - accuracy: 0.8479
 97/100 [============================>.] - ETA: 1s - loss: 0.3654 - accuracy: 0.8485
 98/100 [============================>.] - ETA: 0s - loss: 0.3650 - accuracy: 0.8485
 99/100 [============================>.] - ETA: 0s - loss: 0.3655 - accuracy: 0.8485
100/100 [==============================] - 37s 367ms/step - loss: 0.3650 - accuracy: 0.8485

100/100 [==============================] - 43s 435ms/step - loss: 0.3650 - accuracy: 0.8485 - val_loss: 0.5721 - val_accuracy: 0.7180

Our first model’s performance is not that bad but definitely has room for improvement.

best_epoch <- which.min(history$metrics$val_loss)
best_loss <- history$metrics$val_loss[best_epoch] %>% round(3)
best_acc <- history$metrics$val_accuracy[best_epoch] %>% round(3)

glue("Our optimal loss is {best_loss} with an accuracy of {best_acc}")

Our optimal loss is 0.557 with an accuracy of 0.705

plot(history) + 
  scale_x_continuous(limits = c(0, length(history$metrics$val_loss)))

Scale for 'x' is already present. Adding another scale for 'x', which will replace the existing scale.
`geom_smooth()` using formula 'y ~ x'

CNN with Image Augmentation

Image Augmentation

Our model above does ok but definitely has room for improvement. One approach to improve performance is to collect more data. Unfortunately, this is not always an option. An alternative is to use image augmentation. ℹ️

datagen <- image_data_generator(
  rescale = 1/255,
  rotation_range = 40,
  width_shift_range = 0.2,
  height_shift_range = 0.2,
  shear_range = 0.2,
  zoom_range = 0.2,
  horizontal_flip = TRUE,
  fill_mode = "nearest"
)

The following helps to visualize the idea of image augmentation by:

• Reading in the first image and resizing it to 150x150,
• Converting it to an array with shape (150, 150, 3),
• Reshaping it to (1, 150, 150, 3),
• Generating batches of randomly transformed images.

# get the first cat image
fnames <- list.files(train_cats_dir, full.names = TRUE)
img_path <- fnames[[1]]

# resize & reshape
img <- image_load(img_path, target_size = c(150, 150))
img_array <- image_to_array(img)
img_array <- array_reshape(img_array, c(1, 150, 150, 3))

# generate a a single augmented image
augmentation_generator <- flow_images_from_data(
  img_array,
  generator = datagen,
  batch_size = 1
)

# plot 10 augmented images of the first cat image
op <- par(mfrow = c(2, 5), pty = "s", mar = c(0, 0.1, 0, 0.1))
for (i in 1:10) {
  batch <- generator_next(augmentation_generator)
  plot(as.raster(batch[1,,,]))
}
par(op)

Build & train model

Let’s create a new model that includes image augmentation and we’ll apply the dropout regularization method. The following creates a CNN architecture with:

• Four sequential conv and max pooling layers
• Flatten layer
• Dropout layer
• Densly-connected network

All of which you are familiary with by now.

model <- keras_model_sequential() %>%
  layer_conv_2d(filters = 32, kernel_size = c(3, 3), activation = "relu", 
                input_shape = c(150, 150, 3)) %>%
  layer_max_pooling_2d(pool_size = c(2, 2)) %>%
  layer_conv_2d(filters = 64, kernel_size = c(3, 3), activation = "relu") %>% 
  layer_max_pooling_2d(pool_size = c(2, 2)) %>%
  layer_conv_2d(filters = 128, kernel_size = c(3, 3), activation = "relu") %>% 
  layer_max_pooling_2d(pool_size = c(2, 2)) %>%
  layer_conv_2d(filters = 128, kernel_size = c(3, 3), activation = "relu") %>% 
  layer_max_pooling_2d(pool_size = c(2, 2)) %>%
  layer_flatten() %>%
  layer_dropout(rate = 0.5) %>%
  layer_dense(units = 512, activation = "relu") %>%
  layer_dense(units = 1, activation = "sigmoid")

model %>% compile(
  loss = "binary_crossentropy",
  optimizer = optimizer_rmsprop(lr = 0.0001),
  metrics = "accuracy"
)

Now we can add image augmentation to our image_data_generator(). The rest of the inputs remain the same.

Note:

• Without a GPU this will take approximately 60 minutes to train
• With GPUs this will take approximately 20-30 minutes

# only augment training data
train_datagen <- image_data_generator(
  rescale = 1/255,
  rotation_range = 40,
  width_shift_range = 0.2,
  height_shift_range = 0.2,
  shear_range = 0.2,
  zoom_range = 0.2,
  horizontal_flip = TRUE,
)

# do not augment test and validation data
test_datagen <- image_data_generator(rescale = 1/255)

# generate batches of data from training directory
train_generator <- flow_images_from_directory(
  train_dir,
  train_datagen,
  target_size = c(150, 150),
  batch_size = 20,
  class_mode = "binary"
)

# generate batches of data from validation directory
validation_generator <- flow_images_from_directory(
  valid_dir,
  test_datagen,
  target_size = c(150, 150),
  batch_size = 20,
  class_mode = "binary"
)

# train model
history <- model %>%
  fit_generator(
    train_generator,
    steps_per_epoch = 100,
    epochs = 100,
    validation_data = validation_generator,
    validation_steps = 50,
    callbacks = callback_early_stopping(patience = 10)
  )

As you can see, using image augmentation helps to improve our model’s performance. In fact, if we had more patience with our early stopping we may even be able to nudge out a little more loss reduction.

best_epoch <- which.min(history$metrics$val_loss)
best_loss <- history$metrics$val_loss[best_epoch] %>% round(3)
best_acc <- history$metrics$val_accuracy[best_epoch] %>% round(3)

glue("Our optimal loss is {best_loss} with an accuracy of {best_acc}")

plot(history) + 
  scale_x_continuous(limits = c(0, length(history$metrics$val_loss)))

Save the model

We can always save our models as h5 files. Let’s save this model as we will use it one of the “extras” notebooks to illustrate how we can visualize CNNs (see this notebook https://misk-data-science.github.io/misk-dl/notebooks/99x6-visualizing-what-cnns-learn.nb.html).

model %>% save_model_hdf5("cats_and_dogs_small_1.h5")

However, we still have room for improvement because we are only using a small subset of the available data. We have two options to improve our model:

1. Use more data. We are only using 2,000 of the 25,000 available images. However, this would have a significant impact on compute time.

2. Use transfer learning. This is much quicker than the first option so in the next module I demonstrate how to use transfer learning for CNNs.

Takeways

• When using image data we…
  • use image_data_generator to read the images, decode pixel values, convert to a tensor, rescale, and perform image augmentation.
  • use flow_images_from_directory import batches of our images, apply the image_data_generator, resize, and infer training labels.
• Image augmentation such as zooming, flipping, rotating, shearing, etc. helps with image variance, provides free additional data, and generally improves model performance.

🏠